MEMORANDUM October 5, 2017 
TO: Board Members 


FROM: Richard A. Carranza 
Superintendent of Schools 


SUBJECT: TEACHER APPRAISAL AND DEVELOPMENT SURVEY EVALUATION 
REPORT, 2016-2017 


CONTACT: Carla Stevens, 713-556-6700 


The Houston Independent School District (HISD) Teacher Appraisal and Development System 
(TADS) is intended to improve teacher performance and increase student achievement. This 
report analyzes results from the TADS Evaluation Survey, which assessed teacher and 
appraiser perceptions about TADS in 2016-2017. Perceptions about TADS are presented by 
domain: legitimacy, impartiality, fidelity, value of feedback, quality of feedback, and value of 
formal supports. 


Key findings include: 

e Ofthe 10,875 teachers and 1,055 appraisers eligible to complete the 2016-2017 survey, 33 
percent of teachers and 41 percent of appraisers completed all or some of the survey. 

e Innearly all domains presented in the survey, more than half of teachers and appraisers had 
a positive perception of TADS. However, compared to teachers, appraisers consistently had 
a higher proportion of respondents that viewed TADS positively across all domains. 

e More than half of appraisers (54%) disagreed or were indifferent to the statement that TADS 
requires an appropriate amount of time. 

e For the domain of impartiality for teachers who responded to both the 2014-2015 and 2016- 
2017 surveys, the proportion of teachers that had a positive perception of the TADS system 
to objectively measure teacher effectiveness increased by 11 percentage points (29% to 
40%). 

e Twenty-one percent of teachers reported that they wanted to dispute their rating and/or 
Student Performance measure at some point in time. 


Should you have any further questions, please contact Carla Stevens in Research and 


Accountability at 713-556-6700. 
Tuc 


Attachment 


cc: Superintendent's Direct Reports 
Gloria Cavazos Dawn Randle 
Grenita Lanthan Abby Taylor 
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Teacher Appraisal and Development System: 
Survey Analysis, 2016-2017 


Executive Summary 


Evaluation Description 

Houston Independent School District's Teacher Appraisal and Development System (TADS) is intended to 
promote effective teaching in every classroom, which ultimately translates into improved student 
achievement. TADS Leadership views TADS as a collaborative process that evaluates teacher 
performance, and individualizes support and feedback to positively impact student learning outcomes 
(Leadership Development, 2013). This report analyzes results from the TADS Evaluation Survey for 2016- 
2017. Perceptions about TADS are presented by domain: legitimacy, impartiality, fidelity, value of feedback, 
quality of feedback, and value of formal supports. 


Highlights 


Of the 10,875 Houston Independent School District (HISD) teachers and 1,055 appraisers enrolled in 
the TADS Feedback & Development (F&D) or Student Performance (SP) Tool and eligible to receive 
or give a summative appraisal rating, 33 percent of teachers and 41 percent of appraisers responded 
to the 2016-2017 survey. 


For the domain of legitimacy, most teachers and appraisers agreed or strongly agreed that the TADS 
system provides clear and specific expectations to teachers. However, appraisers appeared to have a 
slightly higher level of agreement, eight percentage points higher, compared to teachers asked the 
same question (75% compared to 67%). 


For the domain of impartiality for teachers who responded to both the 2014-2015 and 2016-2017 
surveys, the proportion of teachers that had a positive perception of the TADS system to objectively 
measure teacher effectiveness increased by 11 percentage points (29% to 40%). 


For the domain of fidelity, teachers’ and appraisers’ opinion of the time required for the TADS system 
varied. The proportion of appraisers that agreed or strongly agreed that the TADS requires an 
appropriate amount of time (46%) was 11 percentage points lower than the proportion of teachers that 
responded to the same question (57%). 


Additionally, for the domain of fidelity, from 2014-2015 to 2016-2017, there was a seven percentage 
point increase (57% to 64%) in proportion of teachers who took the survey both years that had a positive 
perception of the consistency of communication about steps needed to complete Student Performance 
measures. 


For the domain of the value of feedback, new teachers had a more positive perception of value of 
feedback given and received through TADS compared to teachers with more years of experience. In 
their first year, 83 percent of new teachers had a positive perception of the utility of the value of feedback 
through TADS leading to a change in instructional practice. Concurrently, the proportion of teachers 
with a positive perception was 12 percentage points lower for teachers with one to five years of 
experience (71%). 
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For the domain of the quality of feedback, the majority of appraisers (96%) viewed the feedback they 
delivered as useful for teachers’ instructional practice. In contrast, the proportion of teachers that 
perceived the feedback they received from their appraiser as useful for their instructional practice (69%) 
was 27 percentage points lower. 


For the domain of the value of formal supports, the distribution of teachers’ perception of the value of 
formal supports through TADS by accountability rating was nearly identical for Met Standard schools 
compared to Improvement Required (IR) schools. The proportion of teachers that viewed formal 
supports as positive was three percentage points higher for teachers at Met Standard schools (73%) 
compared to teachers at IR schools (70%). 


Twenty-one percent of teacher respondents reported that they wanted to dispute their rating and/or 
Student Performance measure at some point in time (n=735). Of the 204 teachers that elected to 
respond to an open-ended question requesting additional information about their experience with the 
appeals process, 16 percent feared retaliation from their administrator or appraiser (n=33) and five 
percent reported that they were bullied by their administrator or appraiser for expressing an intention 
to dispute their rating (n=10). 


Recommendations 


In nearly all domains presented in the survey, more than half of teachers and appraisers had a positive 
perception of TADS. However, compared to teachers, the proportion of appraisers that viewed TADS 
positively was higher across all domains. TADS Leadership may want to consider ways to improve 
teachers’ perception of TADS to be more consistent with the perception of appraisers, possibly through 
an increase to directed, formal training for teachers. 


While a large proportion of appraisers had a positive perception of TADS across all domains, more than 
half of appraisers (54%) disagreed or were indifferent to the statement that TADS requires an 
appropriate amount of time. When asked about the TADS Timeline, 38 percent of appraisers that 
elected to respond to the open-ended question felt that TADS required too much time (n=38). Future 
research could explore the amount of time that appraisers spend on appraising and supporting teachers 
through TADS in relation to other job duties. 


While there appears to be concerns among teachers and appraisers regarding the impartiality of 
measuring teacher effectiveness through TADS, data shows that the proportion of teachers with a 
positive view of impartiality has improved from 2014—2015 to 2016-2017 for teachers who took the 
survey both years. TADS Leadership should continue efforts to strengthen the implementation and 
transparency of TADS throughout the district. 


Open-ended responses from teachers suggest that a teacher's experience with TADS varies by the 
appraiser to which they are assigned. When asked about the TADS Timeline, 17 percent of teachers 
that elected to respond expressed varying experiences with TADS, either positive or negative, that 
were connected to their appraiser (n=63). TADS Leadership may want to consider ways to ensure the 
uniformity of TADS implementation for all teachers in the district. 


Appraisers and administrators are responsible for communicating a large portion of TADS procedures, 
requirements, deadlines, and feedback. A consistent theme in the teachers’ survey was a request for 
improvements to communication about the TADS Timeline, Student Progress process, and appeals 
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process. TADS Leadership should continue efforts to improve the timeliness and quality of 
communication throughout the school year. 


Administrative Response 


The results of the 2016-2017 TADS Survey provided the Talent Development & Performance team 
with a wealth of information related to TADS implementation. 


During the 2016-2017 school year, 88.4 percent of teachers received an end-of-year Instructional 
Practice rating of Highly Effective (25.7%, n=2,813) or Effective (62.7%, n=6,856). Only 11.5 percent 
of teachers were rated by appraisers as Needs Improvement (10.3%, n=1,128) or Ineffective (1.2%, 
n=136) (TADS F&D Tool, October 2017). Overall, Instructional Practice ratings indicate that appraisers 
are observing effective classroom instruction. 


Results of HISD’s survey align with current nation-wide findings related to teacher evaluations; there is 
room for growth in the observation and feedback process. 


Throughout the survey, results indicate that first year teachers primarily find the feedback and 
development process favorable, while teachers with one to five years of experience possess a less 
favorable viewpoint; however, teachers with six or more years of experience are more likely to indicate 
that they agree or highly agree with statements indicative of positive perceptions of TADS. 


From the 2014-2015 survey to the 2016-2017 survey, responses show an increase in the impartiality 
of TADS and find increased transparency in the Student Performance process regarding the steps 
required. The Talent Development & Performance team, Research & Accountability, and the TADS IT 
Team are continuously brainstorming ways to enhance the Student Performance process for the end- 
user. Planned improvements to the Student Performance tool include a feature to support the goal 
setting process to assist the user with setting goals consistent with the scale of the measure 
(assessment) and an auto-advance feature. Furthermore, as a result of discussions with Curriculum, 
a scoring guide, listing the potential score points for every Pre-Approved District Assessment, will be 
available to all campus administrators and teachers at the start of the goal setting process during the 
2017-2018 school year. Also planned for 2018-2019 is a common scale for all Pre-Approved 
Performance Tasks district-wide. 


There is still a gap in the perceived usefulness of feedback between appraisers (96%) and teachers 
(69%). To overcome this, it is important to further unite campus appraisers and teacher leaders on the 
campus in order to develop mutual understandings about the feedback and development process. 
Achieving this will occur as a result of strategic planning between the Teacher Career Development 
and Talent Development & Performance teams, resulting in work with campuses to bridge gaps. 


Overall, while appraisers and teachers alike expressed increased satisfaction with the system, growth 
is needed in the areas of consistency of implementation and communication. The Performance 
Continuous Improvement Managers (PCIMs) will continue to partner with the leadership of the Area 
School Offices to provide targeted supports to campus leaders. In turn, PCIMs will work closely with 
Teacher Career Development to uncover ways to enhance the TADS process for teachers. 
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Introduction 


Houston Independent School District's (HISD) Teacher Appraisal and Development System (TADS), as 
part of the Effective Teachers Initiative (ETI), is intended to promote effective teaching in every classroom, 
which ultimately translates into improved student achievement. The TADS system is designed to evaluate 
the teaching practices of a diverse group of teachers (Research and Accountability, 2016). A summary of 
the demographic and school-level characteristics of HISD’s teachers and appraiser populations can be 
found in Appendix A (Table A-1 & A-2, pp. 31-32). 


TADS Leadership views TADS as a collaborative process that evaluates teacher performance, and 
individualizes support and feedback to positively impact student learning outcomes (Leadership 
Development, 2013). TADS, similar to other well-designed evaluation systems, incorporates multiple, 
differently-weighted measures of classroom observations and student performance to evaluate teacher 
effectiveness. In the TADS system, effective teaching is determined by three performance criteria areas, or 
appraisal components — Instructional Practice (IP), Professional Expectations (PE), and Student 
Performance (SP).' The weighted sum of the appraisal components is then used to calculate a teacher's 
TADS summative appraisal rating. Further information on the TADS component distribution and weights for 
2016-2017 can be found in Appendix B (p. 33). 


At the beginning of the school year, the TADS system paired one appraiser with multiple participating 
teachers. The role of the appraiser was to coach each teacher towards effective teaching practices. 
Appraisers were tasked with observing teachers in order to provide useful feedback, which was intended 
to improve teaching practices and support the teacher in curriculum planning and professional development 
(Leadership Development, 2013). The stages of the appraisal system in a school year are outlined in the 
Appraisal and Development Timeline. Further information on the Appraisal and Development Timeline for 
2016-2017 can be found in Appendix C (p. 34). Throughout the school year, appraisers were expected to 
conduct at least two 10-minute walkthroughs and two 30-minute formal observations for each teacher 
assigned to them. The appraiser used the IP rubric to assess a teacher’s skills and ability to promote 
learning through classroom observations and walkthroughs. The appraiser used the PE rubric to assess a 
teacher’s efforts to meet objective, measurable standards of professionalism. The appraiser then assigned 
IP and PE scores to the teacher using standardized rubrics and supported the teacher in determining 
outcome measures for SP at the beginning and end of the year. At the End-of-Year Conference, the teacher 
received his/her final IP and PE ratings from the appraiser. Additional information on the rubrics and 
measures used for each TADS component can be found in Appendix D (p. 35). 


The TADS Evaluation Survey was administered to assess teachers’ and appraisers’ perceptions about 
TADS in 2016-2017. This confidential survey was distributed at the end of the 2016-2017 school year via 
email to both teachers and appraisers.” The goal of this report is to describe the survey results of the TADS 
Evaluation Survey for 2016-2017. Perceptions about the TADS are presented by domain: legitimacy, 
impartiality, fidelity, value of feedback, quality of feedback, and value of formal supports. Details on the six 
domains and the method used to calculate the composite score can be found in Appendix E (p. 36). This 
report presents teacher results as compared to appraiser results, reports on teacher perceptions by 


' In 2016-2017, teachers at campuses with Teacher Incentive Fund (TIF4) grants were eligible to incorporate SP into their final 
summative rating. Teachers at non-TIF4 schools were encouraged, but not required, to complete the Goals and Results Worksheets 
for Student Progress (one of the four SP metrics). SP was not incorporated into the final summative rating for teachers at non-TIF4 
campuses. 

? In 2016-2017, most teachers received their final IP rating from their appraiser by April 21, 2017, prior to receiving an invitation to 
complete the TADS survey in May. 
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subgroups, and compares responses to several questions for teachers who took the TADS survey in 2014- 
2015 and again in 2016-2017. Then, it explores the potential impact of teachers’ experiences with aspects 
of TADS on teachers’ perception of TADS as a fair appraisal system. Finally, this report presents 
recommendations and experiences of teachers and appraisers that elected to respond to open-ended 
questions about specific areas of TADS. 


Methods 


This report analyzes survey responses collected from the 2016-2017 TADS Evaluation Survey for teachers 
and appraisers. The report also uses Human Resources roster data to identify teacher-level demographic 
characteristics and school location, and the TADS Feedback and Development (F&D) Tool to identify 
teachers’ IP and PE ratings for the 2016-2017 school year. Data from the 2014-2015 TADS Evaluation 
Survey for Teachers is used for some comparisons. All data used in this report are confidential. 


Teachers and appraisers received an email at the end of the 2016-2017 school year requesting their 
participation in the survey. Teachers and appraisers invited to complete the survey were identified through 
the TADS Feedback and Development Tool. The online survey was open between May 9, 2017 and June 
1, 2017. Appraiser and teacher survey questions and descriptive statistics of each response are presented 
in Appendix F (Table F-1—Table F-30, pp. 37-64). 


e Teachers were identified using the following criteria: 
o To identify job descriptions specific to teachers, the variable Job Function Code was reported as 
TCH, TEA ELEM, TEA PREK, or TEA SEC. 
o To identify salary plans specific to teachers, the variable Salary Plan was reported as RT, VT, 
RO1 or ROS. 


e A teacher was eligible for appraisal if s/ne taught at least 50 percent of the instructional day and was 
actively employed from the beginning of the school year through the end of April of the same year. 
Teachers invited to complete the TADS Evaluation Survey were HISD teachers that were eligible for 
appraisal under TADS and had logged into the TADS F&D or SP Tool a minimum of one time during 
the 2016-2017 school year. 


e Appraisers invited to complete the TADS Evaluation Survey were HISD certified appraisers that had 
logged into the TADS F&D or SP Tool a minimum of one time during the 2016-2017 school year. 


e A teacher's school level was determined by identifying campus-level assignments specific to each 
teacher as indicated in the 2015-2016 HISD District and School Profiles. Teachers located at 
Community Services, HCC Life Skills, EL DAEP, and Beechnut Academy were excluded in this report. 


e Campus accountability ratings were obtained from the Texas Education Agency (TEA) using the Texas 
Academic Performance Reports (TAPR) for 2014-2015 and 2016-2017. 


e Teachers’ years of experience was determined using total teaching experience as verified by the 
Human Resources Information System (HRIS). Teachers were categorized as new teachers (i.e., in 
their first year of teaching), 1-5 years, 6-10 years, 11-20 years, or more than 20 years of experience. 


e Teacher and appraiser responses, when possible, included information such as email addresses 
assigned by HISD, and ratings information was pulled from the TADS F&D Tool and matched to teacher 
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responses. Teacher and appraiser demographic and school-level data were pulled from the HISD 
Human Resources Information System (HRIS) and attached to responses as well. 


Limitations 

Most teachers and appraisers were invited to complete the survey through an email invitation sent directly 
through the survey vendor website. However, the survey vendor software allows respondents to opt out of 
receiving surveys at any time, restricting the survey administrator's ability to send an email invitation to 
complete future surveys. Consequently, a group of teachers (n=96) and appraisers (n=208) from the TADS 
survey population did not receive an email invitation to complete the survey sent directly through the survey 
vendor website. This group of teachers and appraisers was instead invited to complete the survey through 
an embedded weblink sent from Research and Accountability. Their information, while incorporated in 
aggregate frequency tables, could not be linked to demographic and school-level data, and therefore were 
excluded from some data analyses. 


Results 


e The survey response rate for teachers and appraisers increased from 2014-2015 to 2016-2017. Of 
the 10,875 teachers eligible to participate, 33 percent of teachers elected to complete some or all of 
the 2016-2017 TADS Evaluation Survey (N=3,538). The response rate for teachers in 2016-2017 was 
four percentage points higher than the response rate for the 2014-2015 survey (29%). 


e Of the 1,055 appraisers eligible to participate, 41 percent of appraisers elected to complete some or all 
of the 2016-2017 TADS Evaluation Survey (N=435). The response rate for appraisers in 2016-2017 
was 17 percentage points higher than the response rate for the 2014—2015 survey (24%). 


Legitimacy 
What were teachers’ and appraisers’ perception regarding the legitimacy of TADS in 2016-2017? 


e Figure 1 (p. 7) presents questions that measured opinions regarding the perceived legitimacy of TADS. 
For this report, legitimacy is defined as the ability of the TADS system to accurately measure teacher 
effectiveness. Across all questions, the majority of teachers and appraisers had a mostly positive 
perception regarding TADS’s ability to assess teacher effectiveness in the classroom. 


e Most teachers and appraisers agreed or strongly agreed that the TADS system provides clear and 
specific expectations to teachers. However, appraisers appear to have a slightly higher proportional 
level of agreement, eight percentage points higher, compared to teachers asked the same question 
(75% compared to 67%) (Figure 1, Section 1). 


e Teachers and appraisers differed in their opinions that the TADS components reflect effectiveness in 
the classroom. The proportion of appraisers that agreed or strongly agreed with the statement that the 
TADS components reflect effectiveness in the classroom was 13 percentage points higher compared 
to teachers responding to the same question (70% compared to 57%) (Figure 1, Section Il). 


e Most appraisers agreed or strongly agreed (79%) that appraisals were generally accurate reflections 


of teachers’ instructional practices. The proportion of teachers that agreed or strongly agreed (51%) to 
the same statement was 28 percentage points lower compared to appraisers (Figure 1, Section III). 
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e More than half of teachers (58%) and appraisers (58%) agreed or strongly agreed that teachers viewed 
effective teaching as aligned to the 13 instructional practices in the IP rubric (Figure 1, Section IV). 


Figure 1. Teachers’ and Appraisers’ Perception Regarding the Legitimacy of TADS, 2016-2017 
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Source: TADS Evaluation Data, 2016-2017 
Notes: Percentages may not total 100 due to rounding. 
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What was teachers’ overall perception of the legitimacy of TADS by subgroups in 2016-2017? 


e Figure 2 (p. 8) presents the distribution of a composite score that measured teachers’ overall 
perception about the legitimacy of TADS by teacher-level subgroups. Based on their composite score, 
teachers were identified as having a negative, neutral, or positive perception of the legitimacy of TADS. 
Details on the composite score can be found in Appendix E (p. 36). In most subgroups, with the 
exception of Instructional Practice (IP) rating, the majority of teachers had a positive perception of the 
TADS system as being able to accurately measure teacher effectiveness. 


e The distribution of teachers’ perceived legitimacy of TADS by teachers’ IP ratings varied across 
appraisal rating groups. Of the 2,322 teachers that responded to all the questions included in the 
composite measure, 73 percent of teachers with an IP rating of Highly Effective had a positive view of 
the legitimacy of TADS. Conversely, less than half of teachers with a rating of Ineffective or Needs 
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Improvement (89% and 49%, respectively) had a positive perception of the legitimacy of TADS (Figure 


2, Section |). 


Figure 2. Distribution of Teachers’ Overall Perception of the Legitimacy of TADS by Subgroups, 


2016-2017 (N=2,322) 


20% 


6% 


20% 


|. Instructional Practice (IP) Rating Il. School Level 
100% 100% 
80% 80% 
° 67% Eee 2 68% atk Lee 68% 
3 60% 8 60% 
. Cc 
& 40% & 40% 
25% 26% 25% 
20% 24% 20% 23% 25% 
0% 0% 
Ineffective Needs Effective Highly Not Rated Elementary Middle High Combined 
(n=26) Improvement (n=1,351) Effective (n=110) (n=1,252) (n=388) (n=537) (n=145) 
(n=241) (n=594) 
m=Negative Perception mNeither Positive Perception mNegative Perception mNeither m™Positive Perception 
Ill. Campus Accountability Rating IV. Total Years of Experience 
100% 100% 
80% 80% 
© © 63% 65% 65% 
gS 60% D> 60% 
3 = 
: 2 
SG 40% @ 40% 
28% 24% 24% 


0% 


Improvement Met Standard Not Rated 
Required (n=2,021) (n=16) 
(n=285) 


mNegative Perception mNeither mPositive Perception 
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e The distribution of teachers’ perception of the legitimacy of TADS by accountability rating was 
slightly higher for Met Standard schools compared to IR schools. The proportion of teachers with a 
positive perception was five percentage points higher for teachers at Met Standard schools (66%) 
compared to teachers at IR schools (61%) (Figure 2, Section Ill). 


e Newteachers had a more positive perception of the legitimacy of TADS compared to teachers with 
more years of experience. In their first year, 81 percent of new teachers had a positive perception 
of TADS’s ability to accurately measure teacher effectiveness. The proportion of teachers with a 
positive perception was 17 percentage points lower for teachers with one to five years of experience 


(64%) (Figure 2, Section IV). 
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Was there a change in teachers’ perception of the legitimacy of TADS from 2014-2015 to 2016-2017 
for those who took the survey both years? 


e Figure 3 presents one question regarding the perceived legitimacy of TADS for teachers that took the 
TADS survey both years. From 2014-2015 to 2016-2017, the proportion of teachers that agreed or 
strongly agreed with the statement that effective teaching was aligned to the 13 instructional practices 
in the IP rubric decreased by seven percentage points (62% to 55%). In other words, from 2014-2015 
to 2016-2017, there was a slight decrease in the proportion of teachers who took the TADS survey 
both years that had a positive perception of the TADS system to assess teacher effectiveness through 
the Instructional Practice (IP) rubric. 


Figure 3. Aggregate Change in Teachers’ Perception of the 
Legitimacy of TADS, 2014—2015 through 2016-2017 
Teachers at my school agree that effective teaching 
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Sources: TADS Evaluation Data, 2014-2015, 2016-2017; HR Roster File, 2016-2017 
Notes: Percentages may not total 100 due to rounding. 


Impartiality 
What were teachers’ and appraisers’ perception regarding the impartiality of TADS in 2016-2017? 


e Figure 4 (p. 10) presents questions that measured opinions regarding the perceived impartiality of 
TADS. For this report, impartiality is defined as the ability for the TADS system to objectively measure 
teacher effectiveness. Impartiality includes both the perceived objectivity of the appraiser’s rating and 
the perceived neutrality of appraisal ratings across campuses and classrooms. Overall, there was a 
high variation in teachers’ and appraisers’ perception of TADS as an objective evaluation system of 
teachers’ instructional practice. 


e Teachers and appraisers differed in their opinion that the TADS ratings were independent of an 
appraiser’s personal opinion. The proportion of appraisers that perceived TADS ratings to be an 
objective evaluation of teachers’ instructional practice was 23 percentage points higher compared to 
teachers responding to the same question (76% compared to 53%) (Figure 4, Section I). 
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Figure 4. Teachers’ and Appraisers’ Perception Regarding the Impartiality of TADS, 2016-2017 
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Source: TADS Evaluation Data, 2016-2017 
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e Less than half of teachers and appraisers agreed that student achievement and test scores were 
appropriately tied to TADS ratings. Appraisers had a slightly lower proportion of respondents that 
agreed or strongly agreed compared to teachers asked the same question (40% compared to 45%) 
(Figure 4, Section Il). It should be noted that Student Performance was not included in the 2016-2017 
appraisal ratings for most teachers (see Appendix B, p. 33). 


e Less than half of teachers and appraisers agreed that the TADS system effectively accounts for 
differences in student populations. Only 42 percent of teachers and 38 percent of appraisers agreed or 
strongly agreed that the TADS system was neutral across subject areas, schools, and student 
populations (Figure 4, Section III). 


e Appraisers had a substantially stronger perception of TADS as a fair evaluation system compared to 
teachers. Most appraisers agreed or strongly agreed (65%) that TADS is a fair teacher evaluation 
system. In contrast, 45 percent of teachers agreed or strongly agreed that TADS is fair (Figure 4, 
Section IV). 
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What was teachers’ overall perception of the impartiality of TADS by subgroups in 2016-2017? 


Figure 5 presents the distribution of a composite score that measured teachers’ overall perception 
about the impartiality TADS by teacher-level subgroups. Details on the composite measure can be 
found in Appendix E (p. 36). The greatest variation in the distribution of teachers’ opinion about the 
impartiality of TADS was found when analyzed by teachers’ IP ratings. 


More than half of teachers with an IP rating of Highly Effective (55%) and Effective (54%) had a positive 
view of the TADS system’s ability to be impartial. Conversely, the proportion of teachers with an 
Ineffective (38%) or Needs Improvement (39%) IP rating was at least 15 percentage points lower than 
teachers with an Effective IP rating (54%) (Figure 5, Section |). 


Figure 5. Distribution of Teachers’ Overall Perception of the Impartiality of TADS by Subgroups, 


2016-2017 (N=2,311) 
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Source: TADS Evaluation Data, 2016-2017; HR Roster File, 2016-2017 
Notes: Percentages may not total 100 due to rounding. 


The distribution of teachers’ perception of the impartiality of TADS by accountability rating was slightly 
higher for Met Standard schools compared to IR schools. The proportion of teachers with a positive 
perception was five percentage points higher for teachers at Met Standard schools (53%) compared to 
teachers at IR schools (48%) (Figure 5, Section III). 
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Was there a change in teachers’ perception of the impartiality of TADS from 2014-2015 to 2016- 
2017 for those who took the survey both years? 


e Figure 6 presents one question regarding the perceived impartiality of TADS for teachers that took the 
TADS survey both years. From 2014-2015 to 2016-2017, the proportion of teachers that agreed or 
strongly agreed with the statement that the TADS system was fair increased by 11 percentage points 
(29% to 40%). In other words, while less than half of teachers who took the TADS survey both years 
viewed the TADS system as impartial, the proportion of teachers who took the survey both years that 
had a positive perception of the TADS system to objectively measure teacher effectiveness increased 
from 2014-2015 to 2016-2017. 


Figure 6. Aggregate Change in Teachers’ Perception of the 
Impartiality of TADS, 2014-2015 through 2016-2017 
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Sources: TADS Evaluation Data, 2014-2015, 2016-2017; HR Roster File, 2016-2017 
Notes: Percentages may not total 100 due to rounding. 


Fidelity 
What were teachers’ and appraisers’ perception regarding the fidelity of TADS in 2016-2017? 


e Figure 7 (p. 13) presents questions that measured opinions regarding the perceived fidelity of TADS. 
For this report, fidelity is defined as the reliability and consistency of HISD's implementation of the 
TADS system. Overall, there was variation in teachers’ and appraisers’ perception of the perceived 
quality of HISD's implementation of the TADS system. 


e Most teachers and appraisers agreed or strongly agreed that the instructional rubric is clear and easy 
to understand. However, appraisers had a stronger level of agreement compared to teachers asked 
the same question (73% compared to 65%) (Figure 7, Section 1). 


e Teachers’ and appraisers’ opinion of the time required for the TADS system varied greatly. The 
proportion of appraisers that agreed or strongly agreed that the TADS requires an appropriate amount 
of time (46%) was 11 percentage points lower than the proportion of teachers that responded to the 
same question (57%). Conversely, more than half of appraisers (54%) disagreed, strongly agreed, or 
were indifferent to the statement that TADS requires an appropriate amount of time (Figure 7, Section 


Il). 
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Most appraisers agreed or strongly agreed (89%) that administrators at their school are committed to 
improving instructional practice. The proportion of teachers that agreed or strongly agreed (68%) to the 
same statement was 21 percentage points lower compared to appraisers (Figure 7, Section IV). 


Figure 7. Teachers’ and Appraisers’ Perception Regarding the Fidelity of TADS, 2016-2017 
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Source: TADS Evaluation Data, 2016-2017 
Notes: Percentages may not total 100 due to rounding. 


What was teachers’ overall perception of the fidelity of TADS by subgroups in 2016-2017? 


Figure 8 (p. 14) shows the distribution of a composite score that measured teachers’ overall perception 
about the fidelity of TADS by teacher-level subgroups. Details on the composite measure can be found 
in Appendix E (p. 36). Consistent with previous analysis of domains in this report by teacher subgroups, 
the greatest variation in the distribution of teachers’ opinion about the fidelity of TADS was found when 
presented by teachers’ IP ratings. 


The majority of teachers with an IP rating of Effective (73%) and Highly Effective (75%) had a positive 
view of the reliability and consistency of HISD's implementation of the TADS system. In contrast, 
compared to the proportion of teachers with an Effective IP rating (73%) with a positive perception of 
the fidelity of TADS, the proportion of teachers with a Needs Improvement IP rating (53%) was 20 
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percentage points lower and the proportion of teachers with an Ineffective IP rating (835%) was 38 


percentage points lower (Figu 


re 8, Section l). 


Figure 8. Distribution of Teachers’ Perception of the Fidelity of TADS by Subgroups, 2016-2017 


(N=2,316) 
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e New teachers had a more positive perception of the fidelity of TADS compared to teachers with more 
years of experience. In their first year, 79 percent of new teachers had a positive perception of the 
reliability and consistency of the implementation of the TADS system. The proportion of teachers with 
a positive perception was 10 percentage points lower for teachers with one to five years of experience 


(Figure 8, Section IV). 
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Was there a change in teachers’ perception of the fidelity of TADS from 2014-2015 to 2016-2017 for 
those who took the survey both years? 


e Figure 9 presents one question regarding reliability and consistency of HISD's implementation of the 
TADS system for teachers that took the TADS survey both years. From 2014-2015 to 2016-2017, the 
proportion of teachers that agreed or strongly agreed with the statement that that they received 
consistent information about the steps needed to complete Student Performance measures increased 
by seven percentage points (57% to 64%). In other words, from 2014-2015 to 2016-2017, there was 
an increase in proportion of teachers who took the survey both years that had a positive perception of 
the consistency of HISD’s communication to complete the necessary steps for Student Performance 
measures. 


Figure 9. Aggregate Change in Teachers’ Perception of the 
Fidelity of TADS, 2014-2015 through 2016-2017 
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Sources: TADS Evaluation Data, 2014-2015, 2016-2017; HR Roster File, 2016-2017 
Notes: Percentages may not total 100 due to rounding. 


Value of Feedback 


What were teachers’ and appraisers’ perception regarding the value of feedback received and given 
through TADS in 2016-2017? 


e Figure 10 (p. 16) presents questions that measured opinions regarding the perceived value of feedback 
given and received through TADS. For this report, the value of feedback is defined as the overall utility 
of the feedback though TADS that would lead to teachers’ ability to implement change in their 
instructional practice. In other words, the value of feedback is how well that feedback delivered through 
TADS guides improvement of a teacher’s instructional practice. While over half of teachers and 
appraisers agreed or strongly agreed with the questions, there was a notable variation in the distribution 
of teachers and appraisers regarding the value of the feedback they received or gave through TADS. 


e While the majority of teachers and appraisers had a positive perception about the value of feedback, 
appraisers had a substantially stronger view that teachers knew what to do to implement changes after 
receiving feedback. The proportion of appraisers that agreed or strongly agreed that teachers knew 


3 The questions used for the domain of fidelity were not used in the 2014-2015 TADS survey. A question about the fidelity of Student 
Performance, a component of TADS, is used to show changes in the perception of fidelity. 
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what to do to implement changes after they received feedback from their appraiser (93%) was 20 
percentage points higher than for teachers responding to the same question (73%) (Figure 10, Section 


Il). 
Figure 10. Teachers’ and Appraisers’ Perception Regarding the Value of Feedback, 2016-2017 
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e Teachers’ and appraisers’ responses to a perceived change in instructional practice since August 
based on feedback varied. The proportion of teachers that agreed or strongly agreed that they had 
changed their instructional practice based on feedback from their appraiser (68%) was 20 percentage 
points lower than the proportion of appraisers that responded to the same question (88%) (Figure 10, 
Section IV). 


What was teachers’ overall perception of the value of feedback received and given through TADS 
by subgroups in 2016-2017? 


e Figure 11 (p. 17) presents the distribution of a composite score that measured teachers’ overall 
perception about the value of feedback given and received through TADS by teacher-level subgroups. 
Details on the composite measure can be found in Appendix E (p. 36). The greatest variation in the 
distribution of teachers’ opinion about quality of feedback was found when analyzed by teachers’ IP 
ratings. 
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Figure 11. Composite Mean Score of Teachers’ Perception of the Value of Feedback by Subgroups, 
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The majority of teachers with an IP rating of Effective and Highly Effective had a positive view of the 
quality of feedback through TADS (77% and 81%, respectively). In contrast, when compared to 
teachers with an Effective IP rating, the proportion of teachers with an Ineffective IP rating that had a 
positive view (30%) was 47 percentage points lower and teachers with a Needs Improvement IP rating 
with a positive view (51%) was 26 percentage points lower (Figure 11, Section |). 


New teachers had a more positive perception of the value of feedback given and received through 
TADS compared to teachers with more years of experience. In their first year, 83 percent of new 
teachers had a positive perception of the utility of the feedback though TADS leading to a change in 
instructional practice. The proportion of teachers with a positive perception was 12 percentage points 
lower for teachers with one to five years of experience (Figure 11, Section IV). 
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Was there a change in teachers’ perception of the value of feedback received and given through 
TADS from 2014-2015 to 2016-2017 for those who took the survey both years? 


e Figure 12 presents one question regarding the perceived value of feedback through TADS for teachers 
that took the TADS survey both years.* From 2014-2015 to 2016-2017, the proportion of teachers that 
agreed or strongly agreed with the statement that the feedback they received from their appraiser was 
useful for their instructional practice decreased by four percentage points (73% to 69%). In other words, 
from 2014-2015 to 2016-2017, there was a slight decrease in proportion of teachers who took the 
survey both years that had a positive perception of value of feedback through TADS. 


Figure 12. Aggregate Change in Teachers’ Perception of the 
Value of Feedback, 2014-2015 through 2016-2017 
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Sources: TADS Evaluation Data, 2014-2015, 2016-2017; HR Roster File, 2016-2017 
Notes: Percentages may not total 100 due to rounding. 


Quality of Feedback® 


What were teachers’ and appraisers’ perception regarding the quality of feedback received and 
given through TADS in 2016-2017? 


e Figure 13 (p. 19) presents questions that measured opinions regarding the perceived quality of 
feedback received and given through TADS. For this report, the quality of feedback is defined as the 
utility of the feedback as it relates to teachers’ instructional practice. In other words, the quality of 
feedback is how well that feedback delivered through TADS reflects a teacher’s instructional practice. 
Across all questions, there was not alignment between teachers’ and appraisers’ perception of the 
quality of utility of feedback as it relates to instructional practice. 


e Most appraisers (96%) viewed the feedback they delivered as useful for teachers’ instructional practice. 
In contrast, the proportion of teachers that perceived the feedback they received from their appraisers 
as useful for their instructional practice (69%) was 27 percentage points lower than appraisers’ views 
on the feedback they gave teachers (Figure 13, Section II). 


4 The questions used for the domain of the value of feedback were not used in the 2014-2015 TADS survey. A question about the 
overall utility of feedback received from an appraiser is used to show changes in the perception of the value of feedback. 

5 The questions for the domain of quality of feedback were not used in the in the 2014-2015 TADS survey. Teachers’ perception of 
the quality of feedback given and received through TADS cannot be compared from 2014-2015 to 2016-2017. 
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Figure 13. Teachers’ and Appraisers’ Perception Regarding the Quality of Feedback, 2016-2017 
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Similar to the appraisers’ view of feedback, the majority of appraisers (94%) felt that they provided 
teachers with clear instructional expectations. The proportion of teachers that agreed or strongly agreed 
(71%) to the same statement was 23 percentage points lower compared to appraisers (Figure 13, 


Section Ill). 


Teachers’ and appraisers’ opinion of the consistency of feedback between appraisers and 
administrators varied. The proportion of teachers that agreed or strongly agreed that the feedback they 
received from their appraiser was consistent with feedback from administrators (54%) was 29 
percentage points lower than the proportion of appraisers agreeing with the same question (83%) 


(Figure 13, Section IV). 
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What was teachers’ overall perception of the quality of feedback received and given through TADS 


by subgroups in 2016-2017? 


Figure 14 presents the distribution of a composite score that measured teachers’ overall perception 
about quality of feedback given and received through TADS by teacher-level subgroups. Details on the 
composite measure can be found in Appendix E (p. 36). Consistent with previous analysis of domains 
in this report by teacher subgroups, the greatest variation in the distribution of teachers’ opinions about 
quality of feedback was found when presented by teachers’ IP ratings. 


Figure 14. Composite Mean Score of Teachers’ Perception of the Quality of Feedback by Subgroups, 
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The majority of teachers with an IP rating of Effective (75%) and Highly Effective (84%) had a positive 
view of the quality of feedback through TADS. Conversely, when compared with teachers with an 
Effective IP rating, the proportion of teachers with an Ineffective IP rating (27%) was 48 percentage 
points lower and teachers with a Needs Improvement IP rating (43%) was 32 percentage points lower 


(Figure 14, Section |). 


20 


TADS SURVEY ANALYSIS, 2016-2017 


Teachers with one to five years of experience had a slightly more negative perception of the quality of 
feedback compared to new teachers and teachers with more years of experience. Of teachers with one 
to five years of experience, 17 percent of teachers had a negative perception of the quality of feedback. 
When compared to teachers with one to five years of experience, the proportion of teachers with a 
negative perception was four percentage points lower for new teachers (13%) and teachers with six to 
ten years of experience (13%) (Figure 14, Section IV, p. 20). 


Value of Formal Supports 


What were teachers’ and appraisers’ perception regarding the value of formal supports? 


Figure 15 presents questions that measured opinions regarding the perceived value of formal supports 
for improving teacher effectiveness. For this report, the value of formal support is defined as the 
perceived overall utility of formal supports offered through TADS that would lead to teachers’ ability to 
implement change in their instructional practice While over half of teachers and appraisers agreed or 
strongly agreed with the questions, there was variation in the distribution of teachers and appraisers 
regarding the value of formal supports. 


Figure 15. Teachers’ and Appraisers’ Perception Regarding the Value of Formal Supports, 
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e Teachers and appraisers differed in their opinions that the appraiser works with the teacher to set up 
action steps to improve instructional practice. The proportion of appraisers that agreed with the 
statement that they worked with teachers to set up action steps was 21 percentage points higher 
compared to teachers responding to the same question (88% compared to 67%) (Figure 15, Section |, 
p. 21). 


e Mostteachers and appraisers agreed or strongly agreed that the provided formal supports either helped 
to improve instructional practice or support teachers to improve their instructional practice. However, 
appraisers had a slightly stronger level of agreement compared to teachers asked the same question 
(69% compared to 66%) (Figure 15, Section III). 


What was teachers’ overall perception of the value of formal supports through TADS by subgroups 
in 2016-2017? 


e Figure 16 presents the distribution of a composite score that aims to measure teachers’ overall 
perception of the utility of formal supports offered through TADS. Details on the composite score can 
be found in Appendix E (p. 36). 


Figure 16. Composite Mean Score of Teachers’ Perception of the Value of Formal Supports by 
Subgroups, 2016-2017 (N=2,222) 
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e Consistent with previous analysis of domains in this report by teacher subgroups, the greatest variation 
in the distribution of teachers’ opinion about value of formal supports was found when presented by 
teachers’ IP ratings. Teachers with an Effective or Highly Effective IP rating (71% and 79%, 
respectively) had more positive perceptions of the value of the formal supports compared to teachers 
with an Ineffective or Needs Improvement rating (41% and 53%, respectively) (Figure 16, Section |, p. 
22). 


e The distribution of teachers’ perception of the value of formal supports through TADS by accountability 
rating was nearly identical for Met Standard schools compared to IR schools. The proportion of teachers 
that viewed formal supports as positive was three percentage points higher for teachers at Met 
Standard schools (73%) compared to teachers at IR schools (70%). For both Met Standard and IR 
schools, eight percent of teachers had a negative perception of formal supports (Figure 16, Section III). 


e New teachers had a more positive perception of the value of formal supports compared to teachers 
with more years of experience. In their first year, 81 percent of new teachers viewed formal supports 
as positive. The proportion of teachers with a positive perception was 12 percentage points lower for 
teachers with one to five years of experience (69%). For teachers with at six or more years of 
experience (73%), the proportion of teachers with a positive perception was eight percentage points 
lower than that of new teachers and four percentage points higher compared to teachers with one to 
five years of experience (Figure 16, Section IV). 


Was there a change in teachers’ perception regarding the value of formal supports from 2014-2015 
to 2016-2017 for those who took the survey both years? 


e Figure 17 presents one question regarding the perceived value of formal supports through TADS for 
teachers that took the TADS survey both years. From 2014-2015 to 2016-2017, the proportion of 
teachers that agreed or strongly agreed with the statement that they had sufficient support to implement 
changes after receiving feedback from an observation remained unchanged at 67 percent. 


Figure 17. Aggregate Change in Teachers’ Perception of the 
Value of Feedback, 2014-2015 through 2016-2017 
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Notes: Percentages may not total 100 due to rounding. 
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How did teachers’ experience with aspects of TADS align with their perception of TADS as fair? 


e Figure 18 shows the relationship between teachers’ experience with aspects of the TADS system and 
their perception of TADS as a fair appraisal system. The data suggest directional alignment between 
the teachers’ experience with aspects of TADS and the perception of TADS as fair. 


Figure 18. Alignment of Teachers’ Experience with TADS and the Perceived Fairness of TADS, 
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e Teachers that strongly agreed that their appraiser provided clear expectations also had higher levels 
of agreement that TADS is fair. A total of 661 teachers strongly agreed that their appraiser provided 
clear expectations, and of those teachers, 66 percent of them agreed or strongly agreed that TADS is 
fair. Conversely, 201 teachers strongly disagreed that their appraiser provided clear expectations, and 
of those teachers, 73 percent disagreed or strongly disagreed that TADS was fair (Figure 18, Section 


l). 
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e Teachers that strongly agreed that their appraiser provides useful feedback also had higher levels of 
agreement that TADS is fair. A total of 622 teachers strongly agreed that their appraiser provided useful 
feedback, and of those teachers, 68 percent agreed or strongly agreed that TADS is fair. Conversely, 
231 teachers strongly disagreed that their appraiser provides useful feedback, and of those teachers, 
73 percent disagreed or strongly disagreed that TADS was fair (Figure 18, Section Il, p. 24). 


e The greatest alignment of teachers that strongly agreed that TADS was a fair appraisal system was 
found with teachers’ level of agreement with the statement that the TADS system effectively accounts 
for differences in subject areas, schools, and student populations. Of the 192 teachers that strongly 
agreed that TADS effectively accounts for differences in students, 86 percent agreed or strongly agreed 
that TADS is fair (Figure 18, Section Ill). 


e The greatest alignment of teachers that strongly disagreed that TADS was a fair appraisal system was 
found with teachers’ level of agreement with the statement that TADS ratings were independent of 
his/her appraiser’s personal opinion. Of the 291 teachers that strongly disagreed that TADS ratings 
were objective, 83 percent disagreed or strongly disagreed that TADS is fair (Figure 18, Section IV). 


Respondents’ Recommendations 
What were teachers’ and appraisers’ recommendations to improve the TADS Timeline? 


e Of survey respondents, 11 percent of teachers (n=375) and 23 percent of appraisers (n=99) elected to 
respond to a question eliciting recommendations to improve the sequence about the TADS Timeline. 
Open-ended topics and samples can be found in Appendix F (Table F-7-Table F-10, pp. 42-45). 


e Nearly 26 percent of teachers recommended an improvement to the quality of communication about 
the TADS Timeline (n=96). Additionally, 12 percent of appraisers recommended that the End-of-Year 
Conference (EOY) be conducted later in the school year (n=12). 


e Many responses to the question about the TADS Timeline from both teachers and appraisers offered 
general recommendations or opinions about TADS. For teachers, six percent of teachers that 
responded to the open-ended question recommended an increased focus on instructional practice 
rather than accountability (n=21), and 17 percent of respondents expressed that their experience with 
TADS, either positive or negative, was affected by their appraiser (n=63). For appraisers that responded 
to the open-ended question, 10 percent of respondents recommended fewer observations for Highly 
Effective or master teachers (n=10), and 38 percent of respondents felt that the TADS process required 
too much time (n=38). 


What were teachers’ and appraisers’ recommendations to improve the Student Progress process? 


e Of survey respondents, 13 percent of teachers (n=454) and 30 percent of appraisers (n=129) elected 
to respond to a question eliciting recommendations for additional training or supports to assist teachers 
in carrying out the Student Progress® process. Open-ended topics and samples can be found in 
Appendix F (Table F-12—Table F-15, pp. 47-50). 


5 Student Progress is a student learning measure used in the Student Performance component of TADS that uses summative 
assessments or performance tasks to measure how much content and skill students learned over the duration of a course or year, 
based on where they started the subject or course. 
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e The open-ended responses from both teachers and appraisers were similar. Nearly 27 percent of 
teacher respondents requested an increase to the amount of training and support offered about Student 
Progress (n=122). The most commonly requested type of training and support was on how to set goals 
and determine results (7%, n=31)). For appraisers, 43 percent of respondents requested an increase 
to the amount of training and support offered about Student Progress (n=55), and 14 percent requested 
additional training and support on how to set goals and determine results (n=18). 


e Many responses to the question about Student Progress from both teachers and appraisers offered 
general opinions about Student Progress. For teachers, 10 percent of respondents felt that the Student 
Progress measures were not clear or transparent (n=44), and six percent of respondents viewed the 
Student Progress measures as unreliable (n=27). For appraisers, six percent of respondents felt that 
the Student Progress process required too much time (n=8), and five percent of respondents thought 
that the Student Performance tool did not work well (n=6). 


What were teachers’ recommendations and experiences in regards to the appeals process? 


e A question about the appeals process for TADS was presented to teachers. Of survey respondents, 8 
percent of teachers (n=280) elected to respond to a question eliciting general recommendations to 
improve the appeals process. Open-ended topics and samples can be found in Appendix F (Table F- 
19-Table F-20, pp. 53-54). 


e Of the 280 teachers that responded to the question eliciting general recommendations to improve the 
appeals process, 23 percent of respondents recommended that the appeals process be straightforward 
and understandable (n=65) and six percent of teacher respondents recommended that a second 
appraiser should not be informed of the first appraisal rating during the appeals process (n=18). 
Additionally, 18 percent of teachers reported they feared retaliation from their administrator or appraiser 
for disputing a rating (n=51) and more than one percent reported that they were bullied by their 
administrator or appraiser for expressing the intention to dispute a rating (n=4). Another six percent of 
teacher respondents reported they were unaware that disputing a rating was possible (n=16). 


e Of the 3,538 teacher respondents that elected to complete some or all of the TADS Survey, twenty-one 
percent reported they wanted to dispute their rating and/or Student Performance (SP) measure at some 
point in time (n=735). Open-ended topics and samples can be found in Appendix F (Table F-21-Table 
F-22, pp. 55-56). 


e Of the 735 teacher respondents who wanted to appeal their rating or SP measure, 28 percent of 
teachers (n=204) elected to respond to a question requesting additional information about their 
experience with the appeals process. More than 16 percent of teachers that wanted to appeal feared 
retaliation from their administrator or appraiser for disputing their rating (n=33) and more than five 
percent reported they were bullied by their administrator or appraiser for expressing the intention to 
dispute their rating (n=10). Additionally, 12 percent of teachers reported they wanted to appeal but 
ultimately chose not to do so (n=24). 
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Discussion 


The 2016-2017 TADS Survey was offered as a means for teachers and appraisers to share their 
perspectives regarding HISD’s teacher appraisal system. The data gathered in the survey was intended to 
evaluate the TADS system. This report offers information on the perceived strengths and challenges of 
TADS through the analysis of the appraisal system by domain (i.e., legitimacy, impartiality, fidelity, value of 
feedback, quality of feedback, and value of formal supports). 


In nearly all domains presented in the survey, more than half of teachers and appraisers had a positive 
perception of TADS. However, compared to teachers, appraisers consistently had a higher proportion of 
respondents that viewed TADS positively across all domains. One explanation for a higher proportion of 
appraisers with a positive view of TADS is the relationship dynamic of receiving or administering an 
evaluation. While appraisers were held to a high level of expectations to meet TADS deadlines and 
requirements, final appraisal ratings did not directly impact an appraiser’s personal evaluation at the end of 
the year. Teachers, on the other hand, may have experienced higher stakes throughout the year, as they 
were being evaluated and receiving a final appraisal rating. Another possible reason for the variance in the 
perception between teachers and appraisers could be related to the difference in training that teachers and 
appraisers received from the district. Teachers new to the district received several hours of training in the 
summer from TADS Leadership. For teachers already teaching in the district, campus administrators were 
responsible for training teachers, usually prior to the first day of school. Teachers may then have received 
subsequent training and support directly from their appraiser throughout the school year. In contrast, TADS 
Leadership certified school leaders to become appraisers in the TADS system. New appraisers were 
required to attend multiple days of training for certification in TADS’ (Leadership Development, personal 
conversation, May 23, 2017). TADS Leadership may want to consider ways to improve the inconsistency 
between teachers’ and appraisers’ perceptions of TADS, possibly through an increase to directed, formal 
training for teachers. 


Appraisers were responsible for reminding teachers to adhere to TADS deadlines, disseminating 
information/directives from the district in a timely manner, and providing quality, actionable support to 
teachers that are being appraised. While a large proportion of appraisers had a positive perception of TADS 
across all domains, more than half of appraisers (54%) disagreed or were indifferent to the question about 
the amount of time required for TADS. Additionally, 38 percent of appraisers’ open-ended responses about 
the TADS Timeline were related to the amount of time required to evaluate teachers under the TADS 
system. As one appraiser explained, “It takes an inordinate amount of time as an appraiser to complete the 
forms. If you want to leave good comments that can help the teachers grow, you need to spend one to two 
hours for a half an hour observation.” Future research could explore the amount of time that appraisers 
spend on appraising teachers in relation to their other job duties. 


For both teachers and appraisers, survey data showed variation in the perception of impartiality of TADS. 
Impartiality included both the perceived objectivity of the appraiser’s rating and the perceived neutrality of 
appraisal ratings across campuses and classrooms. In their open-ended responses, many teachers 
requested appraiser accountability through confidential feedback about their appraiser or through periodic 
reviews from the district. As one teacher explained, “[l recommend] providing an opportunity for teachers 
to offer feedback on their appraiser's performance in providing a teacher support to facilitate teaching 
effectiveness. This will reduce the need for appeals.” In the area of neutrality of appraisal ratings across 
campuses and classrooms, less than half of teachers and appraisers agreed that student achievement and 
test scores were appropriately tied to TADS ratings, and less than half of teachers and appraisers agreed 


7 In 2016, existing TADS appraisers were required to be re-certified through the district. 
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that the TADS system effectively accounts for differences in student populations. Moreover, in both 
questions tied to neutrality across classrooms and campuses, the proportion of appraisers that agreed that 
TADS was objective was lower compared to teachers asked the same question. TADS Leadership may 
consider further engaging teachers and appraisers in identifying ways to decrease the perceived subjectivity 
of the TADS system. 


One area of notable improvement is teachers’ perception of impartiality over time. Of teachers that took the 
survey in both 2014-2015 and 2016-2017, there was an 11 percentage point increase in the positive 
perception of the fairness of TADS. However, when asked about the TADS Timeline in 2016-2017, 17 
percent of teachers that elected to respond to the open-ended question expressed varying experiences 
with TADS, either positive or negative, that were connected to their appraiser. Moreover, 21 percent of 
teacher respondents in 2016-2017 reported that they wanted to dispute their rating and/or Student 
Performance measure at some point in time, with some reporting that they feared retaliation and a small 
number reporting that they were bullied for expressing a desire to dispute their rating. While concerns 
among teachers and appraisers regarding objectivity of measuring teacher effectiveness through TADS 
exists, it appears that these perceptions may be improving over time. TADS Leadership should continue 
efforts to strengthen the implementation and transparency of TADS throughout the district. Additionally, 
TADS Leadership may want to consider ways to ensure the uniformity of TADS implementation for all 
teachers in the district. 


Across the six domains, the greatest variation in the distribution of teachers’ perceptions by subgroup was 
found by Instructional Practice (IP) rating. The proportion of teachers with Ineffective and Needs 
Improvement IP ratings that had a negative perception of TADS across all domains was higher compared 
to the proportion of teachers with Effective and Highly Effective IP ratings. The higher negative perception 
for these teachers could be associated with several things. First, teachers with lower IP ratings may not 
have received the same quality of supports as other higher-rated teachers, and their negative perception 
may be a reflection of a different experience. Second, teachers with lower IP ratings may have needed 
more support compared to other teachers and they may have felt that they did not receive sufficient support 
throughout the year. Finally, teachers with Ineffective and Needs Improvement IP ratings may have had an 
acutely negative experience that does not reflect the overall implementation process of the TADS system 
or may have been resistant to the feedback and support offered by the appraiser. 


A consistent theme in the teachers’ and appraisers’ responses was a request for improvements to general 
communication. Specific to the TADS Timeline, many teachers requested timely communication about 
deadlines. One teacher wrote, “The appraiser emailed the timeline task a day or two prior to when the task 
needed to be completed, not as a sequenced calendar throughout the year.” Teachers and appraisers also 
requested more communication about the Student Progress® process for teachers. Multiple appraisers 
recommended district-level training on the student performance component for teachers to supplement the 
training that teachers receive directly from their appraiser. As one appraiser suggested, “It would be 
beneficial for teachers to receive a student performance training similar to the session provided to 
appraisers and for teachers to attend student performance training by campus-level and content.” However, 
despite feedback in open-ended responses that there was confusion about the SP measures, the 
perception of HISD’s ability to communicate consistent information on how to complete SP measures has 
improved since 2014-2015. TADS Leadership should continue efforts to improve the timeliness and quality 
of communication throughout the school year. 


8 Student Progress is a student learning measure used in the Student Performance component of TADS that uses summative 
assessments or performance tasks to measure how much content and skill students learned over the duration of a course or year, 
based on where they started the subject or course. 
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This report was intended to evaluate teacher and appraiser perceptions of TADS and to inform HISD 
stakeholders of the strengths and challenges of the TADS system as it was implemented during the 2016- 
2017 school year. Many teachers and appraisers offered constructive and thoughtful suggests to strengthen 
TADS. Despite the existing challenges with the current appraisal system, TADS Leadership should be 
encouraged by the gains made in the proportion of teachers with a positive perception of TADS from 2014— 
2015 to 2016-2017. These findings suggest that efforts to improve the TADS system have been moderately 
successful over time. 
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Appendix A: Demographic Characteristics 


Table A-1. Demographic Characteristics of Teachers: Population and Sample, TADS Evaluation Survey, 
2016-2017 
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Table A-2. Demographic Characteristics of Appraisers: Population and Sample, TADS Evaluation Survey, 
2016-2017 
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Appendix B: TADS Component Distribution, 2016-2017 


The component weights are applied to derive the Summative Appraisal Rating (IP, PE, and SP combined).' 


Teachers with two TADS components (i.e. no Student Performance rating) have the following weights within teachers’ Summative Appraisal Ratings. 
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TIF4 Teachers with three TADS components have the following weights within teachers’ Summative Appraisal Ratings. 
Instructional Practice Professional Expectations Student Performance 
(IP) (PE) SP) TIF4 CAMPUSES 


50% 20% 30% 


Blackshear Elementary School 
Braeburn Elementary School 
Burrus Elementary School 
Codwell Elementary School 
Durkee Elementary School 

Eliot Elementary School 
Fleming Middle School 

Fondren Middle School 

Foster Elementary School 
Garden Oaks Montessori School 
Grissom Elementary School 
Herrera Elementary School 

Law Elementary School 
Looscan Elementary School 
Mading Elementary School 
McGowen Elementary School 
Milne Elementary School 
Montgomery Elementary School 
Pugh Elementary School 

Ross Elementary School 
Southmayd Elementary School 
Sugar Grove Academy Middle School 
23) Wilson Montessori School 


The various types of Student Performance? * measures have different weights within the Student Performance final rating. y 


OANA ALWNH = 


' All TADS components, including Student Performance (SP) measures of Comparative Growth and Student Progress, use a 4-point scale. 
? Teachers must have a minimum of two Student Performance measures to receive a Student Performance rating included in the summative rating. 
° Value-Added is not available for 2016 — 2017, 2017- 2018. 
4 CG is a district measure based on TELPAS and/or STAAR assessments in certain grade levels and subjects. 
5 Student Progress is a student learning measure that uses fwo measures of a) district-wide/pre-approved/appraiser-approved assessments, b) district-wide/pre-approved/appraiser- 
approved performance tasksAvork products, or c) student attainment (Pre-K teachers only). 
Last updated 04/18/2017 


Source: TADS Student Performance Guidebook, 2016-2017 
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Appendix C: TADS Timeline for 2016-2017 
2016-2017 District TADS Calendar 


Feedback and Development Timeline 


Goal-setting Progress End of Year 


(@xveyqid=jazsaker=: (xe) ahi=sasyqrer=: Conference 
08/18/16 — 10/31/16 Completed by:01/27/17 Completed by 04/21/17 


itatioyaeat=¥@rel-Velsthate pM DI-aailojebantsntantiialladitcoyttop itt 
Begin 08/22/16 & continue throughout the school year 


SU UD Mb e-bbibheler moyen ar-W OMe Wi adabqeyhlopak=pr-hakogi@) of-{=9 qi e- 1a Leyak) 
New Appraiser 09/19/16 Formal appraisal period begins (or 15 days after teacher is 
Certification & Update ibg-bhaK=100) 
Training Teachers mustrecelve a minimum of 2 formal walkthroughs and 2 formal 
New Teacher Training & observations prior to End-of-Year Conferences 
Update Training | (Late hires receive 4 informal walkthroughs with written feedback) 


Development Plans 
IPDPs submitted by 10/31/16; Prescriptive Plans of Assistance developed as needed 


student Performance Timeline 


Measures 


Goals Worksheets Results Worksheets 
Worksheets Completed by teachers Completed by teachers and 


Submitted to teachers by 09/19/16 EVUCU-Yo) sSeonceKe hig approved by appraisers by 


iceman ca ts lancaiis appraisers by 10/31/16 05/31/17 


semester A Courses ; semester B Courses 
Goals Worksheets completed by 10/17/16; Goals Worksheets completed by 02/3/17; 
Results Worksheets completed by O1/6/17 Results Worksheets completed by 05/31/17 


Source: TADS Leadership, personal communication, July 26, 2017 
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HISD Teacher Appraisal and Development System, 2016-2017 


Measure 


Summative 
Rating 
Weight 


Criteria for Measurement 


Planning (PL) 


Instruction (I) 


50% 
OR 


70% 


PL-1 Develops student learning goals 


PL-2 Collects, tracks, and uses student data to drive 
instruction; 


PL-3 Designs effective lesson plans, units, and 
assessments 


| -1 Facilitates organized, student-centered, objective-driven 
lessons 


I-2 Checks for student understanding and responds to 
student misunderstanding 


|-3 Differentiates instruction for student needs by employing 
a variety of instructional strategies 


I-4 Engages students in work that develops higher-level 
thinking skills 


1-5 Maximizes instructional time 


1-6 Communicates content and concepts to students 


1-7 Promotes high expectations for students 


1-8 Students actively participating in lesson activities 


1-9 Sets and implements discipline management 
procedures 


1-10 Builds a positive and respectful classroom environment 


Professional 
Expectations 
Criteria 


Professionalism 
(PR) 


20% 


OR 


30% 


PR-1 Complies with policies and procedures at school 


PR-2 Treats colleagues with respect throughout all aspects 
of work 


PR-3 Complies with teacher attendance policies 


PR-4 Dresses professionally according to school policy 


PR-5 Collaborates with colleagues 


PR-6 Implements school rules 


PR-7 Communicates with parents throughout the year 


PR-8 Seeks feedback in order to improve performance 


PR-9 Participates in professional development and applies 
learning 


Student 
Performance 
Criteria 


*For teachers at 
TIF4 campuses 
only. 


Student 
Performance 
(PR) 


30% 


OR 


N/A 


Value-Added not included in 2016 — 2017 summative rating 


Comparative Growth (CG) on TELPAS grades 3-8 or 
STAAR 


Student Progress 
e On districtwide, pre-approved, or appraiser- 
approved assessments 
e On districtwide, pre-approved, or appraiser- 
approved tasks 
e Student attainment (Pre-K only) 


Source: TADS Manual, 2013-2014; TADS Student Performance Guidebook, 2015-2016 
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Appendix E: TADS Survey Domains and Composite Score 


Teacher Appraisal and Development Survey Analysis by Domain 


Range 
1-8 —Negative Perception 
9-12 — Neutral Perception 

13-20 — Positive Perception 


Total Possible Points in a Domain: 
4 questions x 1-5 level of agreement = 
20 points 


Domain Description Questions 


e The TADS system provides clear and specific expectations to teachers. 
e The TADS components (i.e., Instructional Practice, Professional Expectations, Student 
the perceived overall ability of Performance) reflect my effectiveness in the classroom. 
Legitimacy the TADS system to assess e Atmy school, teachers' appraisals are generally accurate reflections of their instructional 
teacher effectiveness practice. 
e Teachers at my school agree that effective teaching aligns to the 13 instructional 
practices in the IP rubric. 
e The TADS ratings are independent of my appraiser's personal opinion. 
the perceived overall ability for e Student achievement and test scores are appropriately tied to TADS ratings. 
Impartiality the TADS system to objectively e The TADS system effectively accounts for differences in subject areas, schools, and 
measure effectiveness student populations. 
e _HISD's teacher appraisal system is fair. 
the perceived overall reliability e The Instructional Practice rubric is clear and easy to understand. 
Fidelity and consistency of HISD's e The TADS system requires an appropriate amount of my time. 
implementation of the TADS e The TADS SP and F&D tools sufficiently allow me to input my data. 
system e Administrators at my school are committed to improving my instructional practice. 
e My appraiser is sufficiently skilled at giving me components of effective feedback. 
Value of the perceived overall utility of e The feedback I receive from my appraiser is useful for my instructional practice. 
Feedback the feedback as it relates to e My appraiser provides me with clear instructional expectations. 
teachers’ instructional practice e When | received feedback on my instructional practice from different administrators at my 
school, that feedback was consistent between administrators. 
e When | received feedback after an observation, | knew what | needed to do to implement 
the perceived overall utility of inGee chiens. 
e When | received feedback after an observation, | had sufficient support to implement 
Quality of inssieedbae ough TADS that those changes. 
Feedback woule fase a sso wade ability 0 e | have changed my instructional practice since August based on feedback | received from 
implement change in their my appraiser(s) 
inetructional pranlige e The TADS system provides feedback to teachers that is specific, actionable, and includes 
resources. 
the perceived overall utility of e My appraiser works with me to set action steps that will improve the quality of my 
Value of formal supports offered through instructional practice. 
Formal though TADS that would leadto | * The TADS system assists in my professional development. 
Supports teachers’ ability to implement e The formal supports that | need to improve my instructional practice are available to me. 
change in their instructional e __ The types of formal supports provided to me have helped me to improve my instructional 
practice practice this year. 


Source: TADS Survey Evaluation Data, 2016-2017 
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Appendix F: TADS Evaluation Data Tables, 2016-2017 


Table F-1. Teachers' Perception of Professional Growth in the 13 Areas of Instructional Practice, 2016-2017 


Strongly Disagree Disagree Neutral Agree Strongly Agree Total 


Teachers Appraisers Teacher Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers 


Ihave room for growth in [my ability)[supporting teachers' ability] to develop 
student learning goals [PL1]. 
Ihave room for growth in [my ability/[supporting teachers’ ability] to collect, 
track, and use student data to drive instruction [PL2]. 

Ihave room for growth in [my ability)[supporting teachers' ability] to design 
effective lesson plans, units, and assessments [PL3]. 

Ihave room for growth in [my ability/[supporting teachers’ ability] to facilitate 
organized, student-centered, objectivedriven lessons [I1]. 

Ihave room for growth in [my ability|[supporting teachers' ability] to check for 
student understanding and respond to student misunderstanding [l2]. 

Ihave room for growth in [my ability|[supporting teachers' ability] to differentiate 
instruction for student needs by employing a variety of instructional strategies 181 6.0 27 74 | 347 11.5] 61 IG || S22 | Woh || iz 10:1) | 1,775)) 58:8" || 2110) | 57-4 | 396" | 13h 31 8.5 | 3,021 100.1 366 =100.1 
[13]. 
Ihave room for growth in [my ability)[supporting teachers’ ability] to engage 
students in work that develops higher level thinking skills [14]. 


163° 5.4 28 76 | 324 10.7] 65 17.7 | 302 10.0] 31 84 /1,881 62.1 | 221 60.1 | 361 11.9] 23 6.3 | 3,031 100.1 | 368 100.1 
182 6.0 41 ile |) xchat) ) lene |) 7 19.3 | 299 | 9.7 32 8.7 |1,814 59.9] 200 544 | 381 126] 24 6.5 | 3,031 99.9 368 100.0 
225 74 38 10.4 | 378 12.5] 71 19.4 | 345 114 | 50 13.6 |1,734 57.3} 190 51.8 | 343 113] 18 49 | 3,025 99.9 367 = 100.1 
187) 6:2: 35 96 | 399 13.2] 84 23.0) 337 11.1 | 47 128 |1,767 584] 186 50.8 | 336 11.1 14 3.8 | 3,026 100.0 | 366 100.0 


222 74 36 98 | 400 132] 79 216] 335 11.1 46 12.6 |1,714 56.7) 189 51.6) 351 11.6 16 44 | 3,022 100.0 | 366 100.0 


214 7A 21 5.7 | 384 12.7 | 60 16.3 | 319 106) 43 11.7 |1,744 57.8) 217 59.0] 358 11.9 | 27 7.3 | 3,019 100.1 368 §=100.0 


Unave room|for growth in Imy ablityisupporiing teachers (ability) maximize 9 "5a7 | 95 || 48) | 132) 410) | 196) 97, | 266 ||| 372) | 123) 47 | 12.91 |1l612| 53:3 || 159) | 4a6 || 943 |4ia)| 14. | 318) | s024)| 10001|| aes) | 10071 
instructional time [I5]. 

Ihave room for growth in [my ability|/[supporting teachers' ability] to 
communicate content and concepts to students [I6]. 

Ihave room for growth in [my ability/[supporting teachers' ability] to promote 
high academic expectations for students [I7]. 

Ihave room for growth in[my ability)/[supporting teachers’ ability] to encourage 
students to actively participate in lesson activities [I8}. 

Ihave room for growth in [my ability|[supporting teachers’ ability] to set and 
implement discipline management procedures [I9]. 

Ihave room for growth in [my ability)[supporting teachers’ ability] to build a 
positive and respectful classroom environment [I10]. 

Source: TADS Evaluation Survey, 2016-2017 

Note: Percentages may not equal 100 due to rounding. 


283 (9.4 40 10.9 | 448 149] 84 229) 349 116) 51 13.9 |1,608 53.4] 177 482 | 326 10.8 15 4.1 | 3,014 100.1 367 = 100.0 
356 11.8) 41 11.1) 465 154) 74 20.1 | 351 116] 37 10.1 11,524 50.5) 196 53.3 | 323 10.7} 20 5.4 | 3,019 100.0 | 368 100.0 
293 9.7 41 11.2 | 460 153] 98 26.7 | 351 11.7 | 44 12.0 ]1,575 523] 169 46.1 | 334 11.1 15 41 | 3,013 100.1 367 = 100.1 


369 «12.2 | 50 13.6 | 411 13.6) 93 253 | 354 11.7) 47 12.8 )1,530 50.7) 162 44.1 | 352 11.7] 15 41 | 3,016 99.9 367 99.9 


461 153] 60 16.3 | 470 156} 89 242) 353 11.7 | 42 11.4 |1,403 466] 160 435 | 327 10.9 17 46 | 3,014 100.1 368 100.0 
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Table F-2. TADS Evaluation Survey Reponse on the Quality of Feedback Received and Given in the TADS System, 2016-2017 


Strongly Disagree Disagree Neutral Agree Strongly Agree Total 


Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers 


My appraiser knows what | need to do to improve my/ instructional 


practice./I know what teachers need to do to improve their instructional 233 8.0 2 0.6 302 «10.4 2 0.6 383 13.1 10 2.8 |1,396 47.9 | 242 688] 601 20.6 96 27.3 | 2,915 100.0 | 352 100.1 
practice. 

My appraiser is sufficiently skilled at giving me components of effective 

feedback./| am sufficiently skilled in giving teachers components of 289) | 919) 2 06 | 262 9.0 W 2.0 305 10.5 | 15 43 |1,299 44.6 | 237 67.5 | 760 261 90 25.6 | 2,915 100.1 | 351 100.0 
effective feedback.* 


The feedback | receive from my appraiser accurately identifies strengths in 
my instructional practice./The feedback I deliver as an appraiser accurately) 260 8.9 2 06 | 278 9.5 1 0.3 303 10.4} 20 5.7 11,314 45.1 | 243 688 ) 761 26.1 87 24.7 | 2,916 100.0 | 353 100.1 
identifies strengths in teachers’ instructional practice. 


The feedback I receive from my appraiser accurately identifies 
weaknesses in my instructional practice./The feedback I deliver as an 
appraiser accurately identifies weaknesses in teachers’ instructional 
practice. 


268 = 9.2 2 06 | 333 11.4 4 el 444 15.2] 19 41205 43212485 70'5) | 6s e201 79 22.4 | 2,915 100.0 | 352 100.0 


The feedback | receive from my appraiser is useful for my instructional 
practice./The feedback | deliver as an appraiser is useful for teachers’ 268 9.2 2 0.6 270 §=9.3 2 0.6 369 «12.7 | 12 3.4 11,285 44.1 | 243 688] 720 24.7 94 26.6 | 2,912 100.0} 353 100.0 
instructional practice.* 


The feedback | receive from my appraiser is framed in the language of the 
13 instructional practices./The feedback | deliver as an appraiser is in the 171 5.9) 2 0.6 is | 5S) 11 3.1 356 12.3 | 28 8.0 |1,473 50.7 | 230 65.7 | 752 25.9 19 22.6 | 2,907 100.1 350 100.0 
language of the 13 instructional practices. 


My appraiser provides me with clear instructional expectations./I provide 


‘ : : : 233 8.0 2 0.6 | 249 85 6 1.7 | 370 12.7) 15 4.3 |1,308 44.9 | 233 666] 756 259] 94 269 | 2,916 100.0 | 350 100.1 
teachers with clear instructional expectations.* 


My appraiser works with me to set action steps that will improve the quality 
of my instructional practice./I work with teachers to set action steps that will | 295 10.1 2 0.6 286 98 10 28 382 = 13.1 31 8.8 |1,260 43.3 | 224 63.6] 688 23.6 85 24.2 | 2,911 99.9 352 100.0 
improve the quality of their instructional practice.* 


The TADS system provides feedback to teachers that is specific, 


‘ ‘ 154 6.2 9 34 325 13.2) 39 134] 503 204) 42 144 |1,234 49.9] 165 56.7] 255 103] 36 12.4 | 2,471 100.0 | 291 100.0 
actionable, and includes resources.(both) 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
*Indicates a proxy question used for a domain. 
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Table F-3. Teachers’ Perception of Appraiser Feedback in the 13 Areas of Instructional Practice, 2016-2017 


Strongly 


Strongly 


Di Neutral Agr 
isagree ite] gree Agree 


Disagree 


My appraiser's feedback strengthened my ability to develop student learning 
goals [PL1]. 

My appraiser's feedback strengthened my ability to collect, track, and use 
student data to drive instruction [PL2]. 

My appraiser's feedback strengthened my ability to design effective lesson 
plans, units, and assessments [PL3]. 

My appraiser's feedback strengthened my ability to facilitate organized, 
studentcentered, objective-driven lessons [I1]. 

My appraiser's feedback strengthened my ability to check for student 
understanding and respond to student misunderstanding [I2]. 


277 10.1 | 300 11.0} 4382 158 |1,219 445 | 512 18.7 | 2,740 100.1 
274 10.0] 288 105] 489 17.8 |1,177 429 |) 513 18.7 | 2,741 99.9 
290 106] 277 10.1} 506 185 |1,155 422 |) 511 18.7 | 2,739 100.1 
278 10.1 | 273 10.0} 501 18.3 ]1,170 42.7 | 520 19.0 | 2,742 100.1 


271 9.9 | 248 9.1 459 168 |1,232 45.0 | 530 19.3 | 2,740 100.1 


My appraiser's feedback strengthened my ability to differentiate instruction for 
student needs by employing a variety of instructional strategies [I3]. 

My appraiser's feedback strengthened my ability to engage students in work 
that develops higher level thinking skills [14]. 

My appraiser's feedback strengthened my ability to maximize instructional time 
[I5}. 

My appraiser's feedback strengthened my ability to communicate content and 
concepts to students [I6]. 

My appraiser's feedback strengthened my ability to promote high academic 
expectations for students [I7]. 

My appraiser's feedback strengthened my ability to encourage students to 
actively participate in lesson activities [I8]. 

My appraiser's feedback strengthened my ability to set and implement 
discipline management procedures [I9]. 

My appraiser's feedback strengthened my ability to build a positive and 
respectful classroom environment [I1 0]. 

Source: TADS Evaluation Survey, 2016-2017 

Note: Percentages may not equal 100 due to rounding. 


288 105] 283 103] 479 17.5 |1,176 43.0 | 512 18.7 | 2,738 100.0 


286 104] 289 105] 480 17.5 |1,164 425 | 522 19.0 | 2,741 99.9 


279 10.2) 266 97 | 510 186 /1,157 423 | 526 19.2 | 2,738 100.0 


284 104] 282 103] 528 19.3 |1,124 41.1 | 519 19.0 | 2,737 100.1 


281 103] 268 98 | 491 17.9 ]1,172 428 | 525 19.2 | 2,737 100.0 


282 103] 269 98 | 481 17.6 /1,184 43.3 | 520 19.0 | 2,736 100.0 


315 115) 264 96 | 5385 195 /1,110 405] 515 188 | 2,739 99.9 


308 811.3 | 264 9.7 | 531 19.4 |1,096 40.1 | 533° 19.5 | 2,732 100.0 
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Table F-4. Appraisers’ Perception of Training and Support to Assist Teachers in the 13 Areas of Instructional Practice, 2016-2017 


Strongly 
Disagree 


Strongly 
Agree 


Disagree Neutral Agree 


| received sufficient training and support to assist teachers in developing 
student learning goals [PL1]. 

lreceived sufficient training and support to assist teachers in collecting, 
tracking, and using student data to drive instruction [PL2]. 

| received sufficient training and support to assist teachers in designing 
effective lesson plans, units, and assessments [PL3]. 

received sufficient training and support to assist teachers in facilitating 
organized, student-centered, objective-driven lessons [I1]. 

| received sufficient training and support to assist teachers in checking for 
student understanding and responding to student misunderstanding [I2]. 

| received sufficient training and support to assist teachers in differentiating 
instruction for student needs by employing a variety of instructional strategies 13 4.1 al 2a Se) 18.4 | 150 46.7] 28 8.7 321 100.0 
1121 

| received sufficient training and support to assist teachers in engaging 
students in work that develops higher level thinking skills [4]. 

lreceived sufficient training and support to assist teachers in maximizing 
instructional time [I5]. 

| received sufficient training and support to assist teachers in communicating 
content and concepts to students [I6]. 

received sufficient training and support to assist teachers in promoting 

high academic expectations for students [I7]. 

| received sufficient training and support to assist teachers in encouraging 
students to actively participate in lesson activities [18]. 


| received sufficient training and support to assist teachers in setting and 
implementing discipline management procedures [I9]. 


14 44 75 234} 60 18.7 | 152 474] 20 6.2 321 100.1 
13 4.1 76 23.7 | 66 20.6 | 139 43.3 | 27 8.4 321 100.1 
13 411 63 19.7] 63 19.7 | 154 48.1 27 8.4 320 100.0 
9 28 56 176] 63 19.8 | 161 505] 30 9.4 319 100.1 


8 25 51 15.9] 62 19.4 | 165 516) 34 106 | 320 100.0 


13 41 77 =©240] 66 206] 1414 439] 24 7.5 321 =100.1 
8 2.5 53 166] 58 18.1 | 167 522) 34 106 | 320 100.0 
9 28 54 16.9] 55 17.2 | 171 534) 31 9.7 320 = 100.0 
10 3.1 69 216] 67 21.0 |) 149 466] 25 78 320 =100.1 
8 25 49 15.3} 57 178 | 177 55.3] 29 9.1 320 =100.0 


iG 2.2 49 153] 62 19.4 | 167 522) 35 109] 320 100.0 


| received sufficient training and support to assist teachers in building a 
positive and respectful classroom environment [I1 0]. 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 


10 3.1 53 166] 61 19.1 | 159 49.7) 37 11.6 } 320 100.1 
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Table F-5. TADS Evaluation Survey Reponse on the Quality of Implementation of the TADS System, 2016-2017 


Strongly Disagree Disagree Neutral Agree Strongly Agree Total 


Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers 


When | received feedback after an observation, | knew what | needed to do 
to implement those changes./When | delivered feedback after an 171 6.3 1 0.3 220 8.2 4 1.3 336 «12.5 16 5.0 |1,322 49.0 | 231 724 | 649 241 67 21.0 | 2,698 100.0 | 319 100.0 
observation, teachers knew what to do to implement those changes.* 


When | received feedback after an observation, | had sufficient support to 
implement those changes./When | delivered feedback after an observation,| 276 10.2 ‘| O13) | 251} 93: 8 25) || 405) | 15:0") 26 8.2 |1,171 43.4) 217 680] 595 22.1 67 21.0 | 2,698 100.0} 319 100.0 
I provided sufficient support for teachers to implement those changes.* 


Ihave changed my instructional practice since August based on feedback | 
received from my appraiser(s)./The instructional practice of teachers | 189 67.0 2 06 | 219 81 7 22 | 445 165 | 31 9.7 |1,334 49.4} 225 70.5] 511 189] 54 16.9 | 2698 100.0 | 319 100.0 
appraise has changed since August based on feedback | have given.* 


When | received feedback on my instructional practice from different 


administaiors al my sehwol thal eedback wasiconsistent between | a26 | 1241| 5 | 46 | 262 | 97 | 17 | 54 | 646 | 240 | a3 | 40.4 |-Se6 |a70 | 208: | 640 || 45 | 179| 59 | ie |S 695 | 1000 || Siz | tone 
administrators./The feedback | delivered on teachers’ instructional practice 


was consistent with feedback from administrators at the school.* 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
*Indicates a proxy question used for a domain. 
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Table F-6. TADS Evaluation Survey Responses on the Appraisal and Development Timeline, 2016-2017 


Idon't 
ves Bh remember/NA feel 


Teachers Teachers Teachers Teachers 


During the 2016-2017 school year, did your appraiser or campus 
administrative team share the Appraisal and Development 
Timeline/Calendar with you? 


1,961 73.0 2,685 100.0 


Were you able to locate the Appraisal and Development 
Timeline/Calendar online? 


64.2 100.0 


UAT 2,676 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 


Table F-7. Teacher Responses to the Open-Ended Item, “What recommendations, if any, do you have to improve the sequence of the 


Appraisal and Development Timeline/Calendar throughout the year?” 


Number of Percent 
Spore’ Responses % 
Recommendations to improve the TADS Timeline: 
Increase the number of walkthroughs and observations 7 1.9 
Decrease the number of walkthroughs and observations 14 3.7 
Improve the timing of walkthroughs and observations 16 4.3 
Increase the amount of time for walkthroughs and observations 4 ‘lel 
Improve the quality of communication about the TADS Timeline 96 25.6 
Communication about the TADS Timeline is sufficient 2 0.5 
Simplify the TADS Timeline 3 0.8 
Not aware of the TADS Timeline 8 2.1 
The TADS Timeline is appropriate 31 8.3 
Recommendations for TADS: 
Allow for appraiser accountability 13 3.5 
Improve the fidelity of TADS throughout the school year 10 2.7 
Improve the quality of TADS training 6 1.6 
Increase focus on instructional practice 21 5.6 
Permit an appraisal from external and/or additional appraiser 3 0.8 
Eliminate current appraisal system 18 4.8 


Opinion about TADS: 

TADS is not designed to measure effectiveness of ancillary or non-core subject teachers 3 
TADS is subjective 8 
TADS requires too much time 5 

The quality of TADS is highly dependent on appraiser 63 16.8 
° My appraiser changed in the middle of the year 2 
4 
4 


. My appraiser did not communicate with me in a timely manner 

° My appraiser was not knowledgeable about TADS La 

e My appraiser was not knowledgeable in my content area 19 5.1 

e My appraiser did not adhere to the TADS timeline 19 onl 

e My appraiser did not provide constructive feedback 10 2.7 

. My appraiser was supportive and knowledgeable 5 iL) 
Other suggestions 19 Sal 
Unable to evaluate response 25 6.7 


Source: TADS Evaluation Survey, 2016-2017 
Notes: “Not applicable” response variations were removed from the total teacher responses for this item (n=228). This question was 
presented to both teachers and appraisers. 
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Table F-8. Sample Teacher Responses to the Open-Ended Item, “What recommendations, if any, do you have to improve the 


FY-YoLUC-Yatex-Mo) Mm aat-W.\o) ole-licr-1i-lave mm BY-\4-)(e) elan-lal am Minit )ilat-y,Or-lCVacer-Lmanlwelelelaveloi Mm dnl -Mn/-t-Larade 


Sample Recommendations to Improve the TADS Timeline 


“{I recommend] less walk-throughs. The calendar or timeline should be set to maximize the teacher's ability to exhibit their teaching skills along 
with their ability to engage their students. Sometimes walk-throughs or observations happen without any consideration of upcoming testing or 
performances.” 

“| believe that after a certain number of years, veteran teachers who perform at a level 4 do not need as many observations. It takes up a lot of 
time for all parties. This is especially true for the appraiser who could be spending more time with newer teachers who need more guidance and 
help. I'm not saying that veteran teachers don't need observations, but that we don't need as many.” 

“| feel it would go smoother so the classroom flows during observation if we knew when our appraiser was coming. When they come if you have 
already begun your lessons it sometimes seems redundant to the scholars when we are having them stop their lesson and repeat and go over 
objectives and the lesson when we have discussed it, finished our "Do Now" "| Do" "We Do" "You Do" and they are in the middle of completing 
the work on the lesson.” 

“Provide more communication. There were a number of instances where my appraiser failed to meet with me on time which resulted in 
rescheduling the meeting.” 

“Anytime there are problems or changes on dates for entering information or data the follow up sometimes is not communicated in a timely 
manner.” 

“Department Chairs and Instructional coaches failed to set time aside to discuss [the TADS Timeline] or set aside time to speak on this in PLCs. 
This should be sent as a district wide email at the beginning of the school year so all teachers are aware of what is going on.” 

“The appraiser emailed the timeline task a day or two prior to when the task needed to be completed, not as a sequenced calendar throughout 
the year.” 

“At beginning of the school year, we are usually informed of specific dates and | pencil those dates in my calendar. I'd like for that to continue 
happening. Oftentimes, the district websites directory is confusing to navigate to find information.” 

“Being relatively new to the district but a veteran teacher, it just seems like there are multiple websites or portals we are supposed to go to 
throughout the year. It's overwhelming and confusing. | think a digital document with clickable links to locations, document, etc. would be 
helpful. Also, it would be helpful to have the due dates of things in that digital document that you could directly add to your Outlook calendar- 
even your personal digital calendar.” 

“It should be on the school faculty calendar share via office365 with all testing dates, all appraisal and development dates, as well as, any other 
important professional development dates.” 

“It would help if timeline updates and reminders were sent throughout the year within a two-week window of the action being due.” 

“| think the timeline is sufficient. It allowed for me to adjust my teaching techniques and maximize instruction to support my students’ academic 
success! 


Sample General Recommendations and Opinions 


“The appraisers should be held to the same standards of those they appraise. If two walkthroughs and observations are required, it would stand 
to reason there should be one of each per semester. Once done, they must submit the paperwork in the TADS tool in order for the teacher to 
make adjustments in areas of deficiency.” 

“[I recommend] mandatory and non-scored "pre-screening" from appraisers to the teacher [on their instructional] practices before their 
observations. Appraisers must visit teacher's classrooms and provide them informal feedback in paper. It will decrease stress levels and 
increase teacher's confidence.” 

“| believe that if we are trying to show growth in one particular or many areas, then maybe there should be PDs available for us to attend that are 
given or lead by others with the rating of a 4 so that we can see and hear the work and language of what our appraisers are looking for out of us 
in the classroom.” 

“It was a blessing and refreshing to receive constructive feedback that was professional and supportive. My appraiser was very knowledgeable 
about Special Education.” 

“This process seems terribly time intensive on the appraiser and requiring the same things for teachers the administrator knows well and are 
consistently excellent teachers with beginning teacher or teachers needing more support seems unproductive.” 

“New teachers need to be properly educated about appraisals. Also, some of the comments made by my appraiser were completely untrue. In 
other cases, when | explained some misunderstandings, | was told, "Well, it's already there and | cannot do anything!" There are many unfair 
inconsistencies in my appraisal document, and | am not happy about it.” 

“[I recommend having an] appraisal person well trained in the content they are observing. For example, do not have an appraisal person review 
a science class(chemistry/physics) if their previous certification is in reading.” 

“My appraiser provided clear and concise feedback on a weekly basis through my lessons plans. That aided me how to meet and exceed 
expectations. | am happy with the guidance and help | received.” 

“There needs to be a feedback form [from the appraiser] given to the teacher immediately after all observations pinpointing positive and 
negative. All negatives should have a suggestion and resource to correct. Getting feedback months later does nothing for my instructional 
practice nor help me strengthen my traits as a teacher. Being given a 3 just so the appraiser has the power to limit 4s until the end of the year is 
not fair.” 


Total Teacher Respondents 375 


Source: TADS Evaluation Survey, 2016-2017 
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TADS SURVEY ANALYSIS, 2016-2017 


Table F-9. Appraiser Responses to the Open-Ended Item, “What recommendations, if any, do you have to improve the sequence 


of the Appraisal and Development Timeline/Calendar throughout the year?” 


Number of Percent 
HEU Responses % 
Recommendations to improve the TADS Timeline: 
The BOY conference should be held earlier in the school year 3 3.0 
The EOY conference should be held later in the school year 12 IZAl 
Improve the quality of communication about the TADS Timeline 3 3.0 
Recommendations for TADS: 
Fewer observations for highly effective or master teachers 10 10.1 
Increase focus on instructional practice 5 5.1 
Increase support from TADS leadership in order to improve support for teachers 6 6.1 
Opinion about TADS: 
The Student Performance measure is unreliable (i.e., subjective, measurement error) 4 4.0 
TADS requires too much time 38 38.4 
The TADS tools are unreliable or too complicated 8 8.1 
Other suggestions 8 8.1 
Unable to evaluate response 2 2.0 


Source: TADS Evaluation Survey, 2016-2017 
Notes: “Not applicable” response variations were removed from the total appraiser responses for this item (n=10). This question was 
presented to both teachers and appraisers. 
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TADS SURVEY ANALYSIS, 2016-2017 


Table F-10. Sample Appraiser Responses to the Open-Ended Item, “What recommendations, if any, do you have to improve the 


sequence of the Appraisal and Development Timeline/Calendar throughout the year?” 


Sample Recommendations to Improve the TADS Timeline 


“The EOY deadline is frustratingly early - there's just about 2 months between the Progress Conference and the EOY, which requires a great 
deal of planning around a very difficult and busy time of the school year. Moving the Progress Conferences to December, the EOY to the end of 
April and making Summative Ratings available sooner than November will help us appraisers provide more meaningful, thoughtful feedback for 
our teachers.” 

“October is too late for the first round of stuff that is due. | also feel like there is no time that the teacher is held accountable for what is on the 
IPDP. It would also be really nice if the conference deadline was at the end of STAAR testing week because that is a time that teachers are able 
to meet when they are not administering testing.” 

“| believe the calendar works fine for our schedule — the only thing that conflicts is state testing and final conferences. If a Dean/Appraiser is 
also the testing coordinator, it causes conflicts. Again, if the system for Goals and Results [worksheets] was reliable. it would be better.” 

“[l recommend] more reminder emails and communication regarding upcoming deadlines and action items.” 


Sample General Recommendations and Opinions 


“| do feel that because the same expectation is required for all teachers, regardless of their performance levels, | miss out on providing as much 
differentiated support to my teachers in need of more attention.” 

“| believe frequent walkthroughs are more effective than formal observations. The steps can be small and actionable for teachers to truly 
implement. Formal observations are more of a check-in, the amount of feedback when you rate several indicators can be overwhelming for 
teachers and may not seem actionable by teachers.” 

“| would like to be able to more clearly see which teachers still have acknowledgments and actions to complete.” 

“| strongly disagree that the TADS system provides effective resources. That, | feel is sorely missing. Also, the TADS system if applied with 
fidelity, can land teachers in the same rating category. | have several teachers who will wind up on a 3, but they range and vary widely in their 
teaching abilities. One could be a very strong 3, almost a 4, and the other a very weak 3, practically a 2 in reality.” 

“In order to do a sufficient job, you must write a very long and descriptive narrative in both the observation and feedback portion of the tool. This 
takes too much time. We have a million and one things to do and should not be slowed down by the tool. The tool should be the vehicle for 
delivery of teacher observation scores, but the dialogue between a teacher and the appraiser should hold the most weight. Drop down boxes for 
the feedback area would be greatly beneficial. Since the descriptors are listed in the rubric, why not use these in a drop down box. The number 
of items mastered or selected from the drop down box would then equate to the score for that area.” 

“It is a massive amount of time that should be spent supporting teachers in the classroom and not completing paperwork. Reduce the number of 
observations and walkthroughs so appraisers can be better coaches.” 

“It takes an inordinate amount of time as an appraiser to complete the forms. If you want to leave good comments that can help the teachers 
grow you need to spend 1 and a half to 2 hours for a half hour observation. This is fine for a new teacher, or teachers who need to grow and 
develop. For master teachers, it is a waste of time. | was an appraiser for ancillary teachers for a couple of years and struggled using the TADS 
system to appraise them because it wasn't reflective of what was going on in the classrooms.” 

“This system is incredibly time consuming and because of that, both teachers and administrators are stretched too thin. There should be a 
tiered approach - highly effective teachers should not be required to have two walkthroughs and two observations. Appraisers should be able to 
make judgment calls in regards to these teachers. A minimum of one walkthrough and one observation, with the option for more instructional 
feedback if an administrator deems appropriate. There should be more flexibility within the system. The Student Progress component should 
be revamped or completely removed. It has never worked, there is no consistency and it is very time consuming. It is demoralizing to be asked 
to do things that take away from student time/instruction when the system does not work.” 

“| was an Instructional Specialist/coach in a previous district that invested time and effort in my knowledge of growing teachers. The TADS is as 
good as the appraiser. Some appraisers do not keep up with current best practices, etc.” 


Total Appraiser Respondents 99 
Source: TADS Evaluation Survey, 2016-2017 
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TADS SURVEY ANALYSIS, 2016-2017 


Table F-11. TADS Evaluation Survey Reponse on the Understanding of Student Performance (SP) Component and Measures, 2016-2017 


Strongly Disagree Disagree Neutral Agree Strongly Agree Total 


Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers 


lunderstand why | do or do not have student performance included in my final 
appraisal rating./I understand why specific teachers do or do not have student 149 5.7 9 29 247 9.5 43 14.0 474 18.2 40 13.0 | 1,341 51.5 166 54.1 392 15.1 49 16.0 | 2,603 100.0 | 307 100.0 
performance included in their final appraisal ratings. 


lunderstand how my Student Performance measures are assigned to me./I 


understand how my Student Performance measures are assigned to teachers. 188 7.2 14 45 326 12.5 48 15.6 427 16.4 36 11.7 | 1315 | 50:5 167 54.2 346 13.3 43 14.0 | 2602 100.0 | 308 100.0 


lunderstand why my Student Performance measures are assigned to me./| 


understand why my Student Performance measures are assigned to teachers. 177 6.8 10 3.3 291 11.2 37 12.1 417 16.1 33 10.8 | 1,347 51.9 179 58.5 363 14.0 47 15.4 | 2595 100.0 | 306 100.0 


lunderstand the specific Student Performance measures that are assigned to 
me./lunderstand the specific Student Performance measures that are assigned | 160 6.2 12 eX) 299 Wlss} 30 Shas 417 16.1 36 11.7 | 1,348 52.0 178 57.8 369 14.2 52 16.9 | 2,593 100.0 | 308 100.0 
to teachers that | appraise. 


| have received consistent information about the steps | need to take to 
complete my/teachers' Student Performance measures (e.g., how to use the 217 8.3 32 10.4 332 12.8 67 21.8 452 17.4 4 13.3 | 1,257 48.3 125 40.6 344 13.2 43 14.0 | 2,602 100.0 | 308 100.0 
TADS tool to select measures, set goals, input results, etc.) 


Ihave successfully recorded my/teachers' Student Performance measures and 


uploaded proper documentation in the SP Tool. 144 5.5 15 49 229 8.8 53 17.2 545 21.0 68 22.1 | 1,305 50.2 132 42.9 377 14.5 40 13.0 | 2,600 100.0 | 308 100.0 


understand where to find available resources, such as the SP Guidebook, that 
provide guidance on student performance. (both) 231 8.9 10 3.3 435 16.8 36 11.7 502 19.3 36 11.7.) 1,114 429 175 57.0 313 12.1 50 16.3 | 2,595 100.0 | 307 100.0 


As a teacher/as an appraiser, | have a comprehensive understanding of the 
Student Progress process (i.e., completing a Goals Worksheet, determining 
student starting points and goals, completing a Results Worksheet, providing 
student assessment outcomes or rating progress ona rubric). 


245 9'5 18 6.0 202 7.9 49 16.2 441 life? 46 152 | 1,173 | 45.6 149 49.3 510 19.8 40 13.2 | 2,571 100.0 | 302 100.0 


My appraiser/the teachers | appraise has/have a comprehensive understanding 
of the Student Progress process (i.e., completing a Goals Worksheet, 
determining student starting points and goals, completing a Results Worksheet, 
providing student assessment outcomes or rating progress ona rubric). 


229 8.9 32 10.6 284 11.0 73 244 405 15.7 51 16.8 | 1,338 52.0 128 42.2 319 12.4 19 6.3 | 2,575 100.0 | 303 100.0 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
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TADS SURVEY ANALYSIS, 2016-2017 


Table F-12. Teacher Responses to the Open-Ended Item, “What type of additional training or support(s), if any, do you think teachers 


need to carry out the Student Progress process?” 


Number of Percent 
ee Responses % 
Recommendations for additional training and supports on Student Progress: 
Create a one page reference guide to complete the Student Progress process 6 13 
Increase the amount of general training and support regarding Student Progress 122 26.9 
Provide training and support regarding Student Progress at the beginning and/or middle of the 26 57 
year 
Facilitate collaboration on Student Progress process with teachers in similar groups/grades 7 1.5 
Provide training and support on how to differentiate by group/grade level 11 2.4 
Increase the amount of training and support on how to set goals and determining results 31 6.8 
Increase the amount of training and support for new teachers 27 5.9 
Provide training to navigate the Student Performance Tool 9 2.0 
Increase the amount of training and support to use Student Progress data to improve 9 20 
instructional practice , 
Opinion about Student Progress: 
The quality of TADS is highly dependent on appraiser 12 2.6 
Increase the focus of Student Progress on supporting instructional practice 12 2.6 
The training and support for the Student Progress process is sufficient 7 1.5 
The Student Performance Tool does not work well 5 1.1 
There is a lack of consistency or communication about Student Progress process 19 4.2 
Student Progress is not clear or transparent 44 9.7 
Teachers are not able to differentiate by group/grade level 18 4.0 
The Student Progress process requires a lot of time 10 2.2 
The Student Progress measure is unreliable (i.e., subjective, measurement error) 27 5.9 
Student Progress should be removed from TADS 11 2.4 
Other types of training and support 12 2.6 
Unable to evaluate response 29 6.4 


Source: TADS Evaluation Survey, 2016-2017 
Notes: “Not applicable” response variations were removed from the total teacher responses for this item (n=122). This question was 
presented to both teachers and appraisers. 
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TADS SURVEY ANALYSIS, 2016-2017 


Table F-13. Sample Teacher Responses to the Open-Ended Item, “What type of additional training or support(s), if any, do you think 


teachers need to carry out the Student Progress process?” 


Sample Recommendations for Additional Training and Supports on Student Progress 


“{I recommend] proper training or information as to what measures will be measured. This is information that teachers should track throughout 
the year, and by doing, so provide the teacher with true data. Trying to set the goals then input the measures at the same time is probably not 
the best steps to take.” 

“Be consistent. It changes every year and throughout the year. A determination on student measures should be made before the start of our 
duty time as teachers so that we can be properly trained and implement it. Do not change it mid-year. | am still unsure if student progress 
measures are even a part of my appraisal this year.” 

“I recommend] appraisers consistently checking and supporting teachers will be helpful with carrying out the Student Progress process since it 
is usually teachers approaching others for assistance.” 

“| want a training that explains mathematically how this will impact my evaluation and how each student's score is weighed in my evaluation.” 
“Training should be provided about what district tasks are and when/why they are assigned. Additionally, what does a teacher do if the progress 
measure for students reflect something that does not happen? One example from my student progress measure is a district assessment for IB 
Diploma level students when one does not exist. Also, training so teachers know how to save the material in another place for when the system 
inadvertently erases their submissions (which happens occasionally).” 

“A team member from the district office should provide training to teacher leaders and those teacher leaders should have small group training for 
the aforementioned processes. | am still confused about the process and | am entering my three year in the district.” 

“New teachers at my campus were lost. | have done this many years, and | was still confused as how to appoint the measures. [There] needs a 
clear and consistent way to communicate the steps to take when creating these goals and the steps to take after. Some teachers hadn't created 
goals since the beginning of the year. Also, accountability needs to happen at administrative levels also. We depend on them to disseminate that 
information to us teachers.” 

“| would like to be trained on the specifications of the Student Performance tool. | would like to learn how to use this system to track student 
progress.” 


Sample General Opinions about Student Progress 


“My appraiser gave me inaccurate information and stated the student performance would bring my instructional rating up from a 2 to a 3, however, 
student performance is not even included in the instructional piece this year.” 

“The process and expectations are not set ahead of time to allow for proper planning--it all seems to simply happen when the time comes, which 
causes a sense of urgency and disallows for the opportunity to process: it all effectively and efficiently.” 

“The idea of using a system is OK. We need something to measure performance. What | think is: there is no time to digest the information, 
appraisers are in a hurry to turn in evaluations at a due day, observations are not done with taking the time to know and understand the type of 
students you have and what all that involves when it comes to performance. Yes, expectations need to be in place and high expectations, but 
the realistic part of what type of students you have and what kind of knowledge students have or not have brings is a huge deal too.” 

“This entire process is statistically meaningless. Teacher-assigned starting points lack precision or rigor, and none of our assessments (with the 
possible exception of the AP exams) are designed for measuring the kinds of student progress this measure is supposed to track. If the district is 
going to continue to use this measure (which I'm not sure is a good idea at all), it needs to put in the time and money to develop consistent, 
rigorous beginning-of-course and end-of course assessments to track student growth that are designed specifically for that purpose, for every 
course in the curriculum.” 

“TADS does not reflect the specific objectives that the ancillary groups needs to have in order to fulfill this requirement. Nobody knew how to do 
students measures as a district for specific ancillary classes like they do for regular classes. For regular teachers, the test is already provided 
for them, ancillary teachers needed to make their own. Why is it important to show growth in ancillary when ancillary is a mean to broaden 
students’ knowledge. It shouldn't be based on a student written test.” 


Total Teacher Respondents 454 


Source: TADS Evaluation Survey, 2016-2017 
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TADS SURVEY ANALYSIS, 2016-2017 


Table F-14. Appraiser Responses to the Open-Ended Item, “What type of additional training or support(s), if any, do you think 


teachers need to carry out the Student Progress process?” 


Number of Percent 
enSweh Responses % 
Recommendations for additional training and supports on Student Progress: 
Increase the amount of general training and support 55 42.6 
Increase the amount of training and support on how to set goals and determine results 18 14.0 
Opinion about Student Progress: 
Increase the focus of Student Progress on supporting instructional practice 5 3.9 
There is a lack of consistency or communication about Student Progress process 6 4.7 
Student Progress is not clear or transparent 6 4.7 
The teachers that | appraise had a positive experience with the Student Progress process 3 2.3 
The Student Performance Tool is too complicated 6 4.7 
The Student Performance Tool does not work well ih 5.4 
Teachers are not able to differentiate by group/grade level 6 4.7 
The Student Progress process requires a lot of time 8 6.2 
The Student Progress measure is unreliable (i.e., subjective, measurement error) 6 4.7 
Student Progress should be removed from TADS 3 2.3 


Source: TADS Evaluation Survey, 2016-2017 
Notes: “Not applicable” response variations were removed from the total appraiser responses for this item (n=12). This question was 
presented to both teachers and appraisers. 
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TADS SURVEY ANALYSIS, 2016-2017 


Table F-15. Sample Appraiser Responses to the Open-Ended Item, “What type of additional training or support(s), if any, do you think 


teachers need to carry out the Student Progress process?” 


Sample Recommendations for Additional Training and Supports on Student Progress 


“It would be beneficial for teachers to receive a student performance training similar to the session provided to appraisers and for teachers to 
attend student performance training by campus level and content.” 

“[I recommend] additional training and/or support in determining student starting points and goals. Overall, this is the one area that teachers on 
our campus don't fully understand. As an appraiser, | would like additional training in this area as well.” 

“Teachers would like to know the formula used to determine student progress. Teachers should receive district-level face to face training on 
student performance and be provided clear student performance expectations from the district.” 

“New teachers need additional training throughout the year to understand the Student Progress System. The initial training the new teacher 
orientation is not enough. They need follow up training. The process seems to change from year to year and communications from PCIMs 
seems to vary, some are great at communicating and others you only hear from 3 times a year.” 

“Teachers need a more hands-on training about the Student Progress process. It needs to be tailored by grade level or teaching assignment 
because this can vary widely. | think what is most confusing for many teachers is not knowing the different options, and how they were selected. 
Also, it is sometimes very confounding to pigeon hole someone into the measures made available for them. We had several teachers for whom 
we had to assign measures that don't accurately reflect what they teach. ...Also, the whole appraiser approved assessment thing can be a 
daunting process, and it seems a little unfair the way | see it because other teachers have no say in what measures get applied to them. With 
the appraiser approved assessments, and a campus that can be a little lax with those, the level of challenge from BOY to EOY assessments, 
can vary greatly, and that's not fair for those other teachers who have arduous measures to meet.” 

“[Teachers] need hands on practice. They also need a webinar available as a follow up when completing the Student Progress process. 
Although a training is given to them, appraisers end up meeting with the teachers individually to complete the Student Progress measures 
process. This is extremely time consuming, considering that this is done at the beginning of the year and at the end of the year. By the end of 
the year, the teachers have forgotten about most aspects of the Student Progress process.” 

“Ancillary teachers (music, art, physical education, sports, technology, drama, etc.) need training on Student Performance tasks, rubrics, 
tracking systems, etc.” 

“Teachers in Self Contained Special Education classes need additional guidance to carry out the Student Progress process. Given the fact that 
each student has specific goals outlined within their IEPs and differentiated curriculum, teachers really need to be able to create a goals 
worksheet per student.” 


Sample General Opinions about Student Progress 


“The administrators need the messaging to be consistent. When we asked repeated questions during the training last summer we received 
different answers during the course of the same session. Then, the assessment of the training was so out of alignment with the information 
given that the trainers had to give us the answers in order for us to pass. The training and presenters deserved 1s using the TADS tool. The 
most frustrating thing about HISD is TADS! Especially the student progress side.” 

“Some of these [survey] questions are misleading. There has not been a year in HISD when the SP tool worked! The failure of the tool caused 
frustration, confusion, ambiguity, and lack of credibility. The tool even calculated teacher's evaluation score incorrectly after a policy was 
changed, yet the tool could not be adjusted. It created a huge drain on human resources. My answers to the previous questions would have 
been very different had the tool worked properly.” 

“| think the teachers need more information about this system, and how their measures were chosen. | also don't think it is totally fair to have 
some teachers get to choose tasks and rubrics and others are just based on state testing, it seems very subjective.” 

“The student progress component is not an effective use of time as it is currently configured. It needs to be completely changed for all of the 
non-standardized test measures. The system has too many details, is too easy to manipulate, and is too cumbersome.” 


Total Appraiser Respondents 129 


Source: TADS Evaluation Survey, 2016-2017 
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TADS SURVEY ANALYSIS, 2016-2017 


Table F-16. TADS Evaluation Survey Reponse on Personal Experience with the TADS System, 2016-2017 


Strongly Disagree Disagree Neutral Agree Strongly Agree Total 


Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers 


The TADS system provides clear and specific expectations to 


i 127) 5 7 24 | 259 105} 29 10.0 | 433 175] 36 12.4 |1,374 55.5 | 176 60.5 | 282 11.4) 43 14.8 |2,475 100.0} 291 100.1 
teachers.(both) 


The TADS system provides feedback to teachers that is specific, 


A 6 a 154 6.2 g) 3.1 325 13.2] 39 134) 503 204) 42 144 |1,234 499 | 165 56.7 | 255 103) 36 12.4 |2,471 100.0} 291 100.0 
actionable, and includes resources. 


The Instructional Practice rubric is clear and easy to understand.(both)* 120 49 10 3.5 | 241 98 | 34 11.7 | 487 19.7] 37 12.8 |1,314 53.3 | 161 55.5 | 305 124) 48 166 |2,467 100.1} 290 100.1 


The TADS components (i.e., Instructional Practice, Professional 
Expectations, Student Performance) reflect my effectiveness in the PNG hI7 10 34 | 334 135) 34 116) 514 208) 45 15.4 |1,147 464) 166 56.7 | 263 106) 38 13.0 }2,473 100.0] 293 100.1 
classroom. (both)* 


The TADS system assists in my professional development./The TADS 


ae ; : 212 86 1 3.8 | 336 13.6) 49 168 | 528 214] 55 188 |1,138 46.0] 157 53.8 | 259 105] 20 68 |2,473 100.0} 292 100.0 
system assists in teachers’ professional development. 


The TADS ratings are independent of my appraiser's personal 
opinion./The TADS ratings are independent of my personalopinionasan | 297 12.1 3 1.0 | 278 11.3] 22 76 | 583 23.7] 47 162 |/1,032 41.9) 147 50.5 | 274 11.1 | 72 24.7 |2,464 100.1} 291 100.0 
appraiser.” 


Student achievement and test scores are appropriately tied to TADS 


. ts 295 12.0] 32 11.0] 371 15.1] 65 223) 701 285] 77 264 | 877 35.7] 103 353 | 216 88 15 5.1 |2,460 100.1] 292 100.1 
ratings. (both) 


The TADS system effectively accounts for differences insubjectareas, / 547 164 | 33 130| 420 17.0| 75 25.7 | 607 246| 69 236 | 848 344] 96 329/195 79 | 14 48 |2467 1000] 292 1000 
schools, and student populations. (both) 

The TADS system requires an appropriate amount of mytime/The TADS | 546 37 | 71 246] 261 106] 55 19.0| 605 245] 30 104 |1,153 467] 95 329]236 96] 38 13.1 )2,471 1000] 289 100.0 
system requires an appropriate amount of my time as an appraiser. 

The TADS SP and F&D tools sufficiently allow me to input my data./The 
TADS SP and F&D tools sufficiently allow me to access and contribute to | 190 7.7 21 7.2 | 245 99 | 52 179] 732 29.7] 58 19.9 |1,093 44.3 | 142 488 | 206 84 | 18 62 |2,466 100.0] 291 100.0 
teachers’ data.* 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
*Indicates a proxy question used for a domain. 
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TADS SURVEY ANALYSIS, 2016-2017 


Table F-17. TADS Evaluation Survey Reponse on TADS System in Context of School and District, 2016-2017 


Strongly Disagree Disagree Neutral Agree Strongly Agree Be} | 


Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers Teachers Appraisers 


At my school, teachers' appraisals are generally accurate reflections of 


is f : 309 12.8 3 1.0 | 363 15.0] 22 76 | 496 205] 36 124 | 971 40.2 | 184 63.2 | 276 114] 46 15.8 /2,415 99.9] 291 100.0 
their instructional practice. (both) 


Lene ee ware toimproving myinstructional | 555 97 | 5 17/199 79| 5 17 | 347 144] 22 76 |1,182 489] 138 47.4 | 462 19.1 | 121 4161/2416 100.0| 291 100.0 


Teachers at my school share a common vision of effective teaching. (both) | 172 7.2 6 21 | 237° 99 | 48 165] 371 154] 40 13.8 |1,167 485] 153 526 | 460 19.1 | 44 15.1 |2,407 100.1) 291 100.1 


Heachers al my echool/agies thet effective teaching aligie 1p the 13 1e2 67 | 5 17 | 222 92] 39 135| 621 258| 78 271 |1,058 440] 135 469 | 343 143| 31 108|2,406 100.0] 288 100.0 
instructional practices in the IP rubric. (both) 


HISD's teacher appraisal system is fair. (both)* 346 14.3 9 3.1 | 402 166) 46 158) 595 246) 47 162 | 913 37.7] 161 55.3 | 167 69 | 28 9.6 |2,423 100.1} 291 100.0 


HISD's teacher appraisal system is rigorous. (both) 136 56 4 14 | 139 | 57 | 18 | 62 | 612 | 253 | 42 | 145 |1,209) 50.0 | 174 | 60.0 | 324 | 13.4 | 52 | 17.9 |2,420)100.0| 290 | 1000 


The expectations for effective teaching are clearly defined at my school 


through the Instructional Practice rubric. (both) 192 | 7.9 5 1.7 | 234 97 | 26 9.0 | 491 203] 46 15.9 /1,208 50.0) 170 586 | 292 12.1) 43 14.8 |2,417 100.0} 290 100.0 


HISD's culture and climate allow educators to contribute to joint decision- 
making. (both) 

Source: TADS Evaluation Survey, 2016-2017 

Note: Percentages may not equal 100 due to rounding. 

*Indicates a proxy question used for a domain 


400 | 166 | 25 | 87 | 377 | 15.7 | 39 | 1395 | 546 | 227 | 64 | 222 | 884 | 36.7 | 139 | 48.1 | 202) 84 | 22 | 7.6 |2409)100.1) 289 | 100.1 
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Table F-18. TADS Evaluation Survey Reponse on TADS System in Context of School and District, 2016-2017 


Strongly 
Disagree 


Strongly 


Total 
Agree ek 


Disagree Neutral Agree 


Teachers Teachers Teachers Teachers Teachers Teachers 


Sse 49.0 | 240 10.0 |2,405 100.0 
LN Te) 


Total 


lunderstand the necessary steps | would need to take to request a second 
appraiser. 


Yes 


Teachers Teachers Teachers 


At any point in time, have you wanted to dispute your ratings and/or 1,694 69.7 |2,429 100.0 
Student Performance measures? 
Did you feel that you were supported by administrative leaders throughout 


the appeals process? 458 66.4 | 690 


100.0 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 


Table F-19. Teacher Responses to the Open-Ended Item, “What recommendations, if any, do you have to improve the appeals 


je) olex-x-t-¥ (“We Bc =Le [U(=X-) U1 ale r--X-(exe) ale Wr] e) e) ¢-1(-1-1 9 a 


Number of Percent 
Bne wes Responses % 
Suggestions to improve the appeals process: 
Appeals process should be straightforward and understandable 65 23.2 
Instead of requesting a second appraiser, more professional development 4 1.4 
Provide a second appraiser with no knowledge of first appraisal 18 6.4 
Experience with the appeals process: 
Administration or appraiser was supportive and professional 7 2.5 
Appeals process was stressful or cumbersome 7 2.5 
Appeals process was too short (e.g., ran out of time) 5 1.8 
Experienced bullying for expressing a desire to dispute rating 4 1.4 
Feared retaliation from administration or appraiser for disputing rating 51 18.2 
Request to appeal rating was ignored 7 2.5 
Unaware that disputing a rating was possible 16 5.7 
Experience with TADS: 
Appraisals are subjective: administration or appraiser 32 11.4 
Appraisals are subjective: student population 6 21 
Appraiser did not adhere to timeline 2 0.7 
The quality of TADS is highly dependent on appraiser 13 4.6 
Have not had need to appeal rating 6 2.1 
If a second appraiser is granted, then scores are unfairly averaged together 3 1.1 
TADS is not designed to measure effectiveness of ancillary or non-core subject teachers 6 2.1 
TADS requires too much time of administrators or appraiser 1 0.4 
Unable to evaluate response 27 9.6 


Source: TADS Evaluation Survey, 2016-2017 
Notes: “Not applicable” response variations were removed from the total teacher responses for this item. (n=144). This question was 
presented to teachers only. 
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Table F-20. Sample Teacher Responses to the Open-Ended Item, “What recommendations, if any, do you have to improve the appeals 


process (e.g. requesting a second appraiser)?” 


Sample Suggestions to Improve the Appeals Process 


“It should be an open and honest process; teachers should not have to search for how to file an appeal.” 

“| would say that this information needs to be disseminated at any training about the Teacher's Appraisal System, and also when the deadlines 
are being discussed.” 

“{I recommend] providing the opportunity for teachers to offer feedback on /appraise their appraiser’s performance in providing teacher support 
to facilitate teaching effectiveness. This will reduce the need for appeals.” 

“| recommend that the second appraiser should neither be the principal or School Support officer, but a Specialist from the district from that 
content area that is not assigned to the school to have a fair appraisal to be able to compare.” 

“Need to find a way to make sure that 3rd party independent is used when requesting a second appraiser, not have a local campus administrator 
who would give a similar rating to the disputed administrator that share the same agenda is not the way to handle things.” 

“| believe that it would be beneficial to have the name and phone of the first person in command share with teachers so that they know how to 
request a second appraiser or initiate an appeal process.” 

“When many or most of the teachers disagree with their appraiser ratings, or the way he/she interacts with teachers, someone outside school 
should be monitoring these appraiser actions. Constructive feedback is way more effective than demeaning or derogatory criticism towards 
teachers.” 

“| recommend that the second appraiser should neither be the principal or School Support officer, but a Specialist from the district from that 
content area that is not assigned to the school to have a fair appraisal to be able to compare.” 

“Teachers should be explicitly need to be notified of the appeals process. From the first meeting with their appraisers. There should be a system 
in place when a teacher can request a second appraiser. This system gives too much power to appraiser.” 


Sample Experiences with the Appeals Process or TADS 


“| understand the process. However, administrators at my school as of the second week of January 2017 are committed to improving our 
instructional practice, tremendously. There was nowhere | could express my appreciation for the new administration team. They need to be 
commended.” 

“The effectiveness of the appraisal system is entirely dependent on whether or not teachers have a good appraiser. My appraisers this year 
were excellent (rigorous, specific, critical) and so | don't really have objections to how the system has treated me. There isn't continuity across 
appraisers about how to evaluate teachers, however, and bad administrators screw up the ratings for everyone. Substantive point inflation in 
appraisals.” 

“Teachers should be allowed to request a second appraiser twice a year, per semester. Additionally, the final rating should not be an average of 
the initial appraisal and the second appraisal because the teacher is requesting a second appraisal do to the initial appraisal's bias or inaccurate 
rating of the teacher.” 

“The appeals process needs to be distributed earlier for principals that complete the end of year ratings early. | did not receive the email in 
enough time to appeal before the end of my 10 day process” 

‘We don't know how to appeal. There is not a training or pdf or video with step by step instructions. We tend to have to reach out to teacher 
unions who also are uneducated on the process” 

“No one explained that there was an appeals progress at all. This is not discussed at any trainings. It needs to be discussed and must be 
included in all trainings.” 

“It would be helpful if there was enough time for a teacher to discuss the matter with the principal before making the decision to request a 
second appraiser. There is fear of retaliation for requesting a second appraiser.” 

“| disagreed with an observation and feedback but | saw four instances when my co-workers tried to appeal by submitting evidence, asking for 
another appraiser and asking for another observation their requests were ignored by our appraiser and when they did get reappraised it was by 
the same appraiser and even harsher. It made me feel like would not only be a waste of time to ask for another opinion but it would single me 
out for retribution. It created a very defeated atmosphere in our school and a lot of the teachers left this year.” 

“| work at X High School. The principal refused to let me have a second appraiser because in his words he "did not want to establish that 
precedent." 


Total Teacher Respondents 280 


Source: TADS Evaluation Survey, 2016-2017 
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Table F-21. Teacher Responses to the Open-Ended Item, “Please use the space below to add any additional information about your 
experience with the appeals process that you think would be helpful to improving the process.” 
Number of Percent 
feels Responses % 
Suggestions to improve the appeals process: 
Instead of requesting a second appraiser, more professional development 2 1.0 
Provide a second appraiser from with no knowledge of first appraisal 10 4.9 
Experience with the appeals process: 
The administration or appraiser was supportive and professional during the process 10 4.9 
The appeals process was stressful or cumbersome 4 2.0 
The appeals process was not transparent or clear 4 2.0 
Appeals process was too short (e.g., ran out of time) 3 1.5 
Experienced bullying for expressing a desire to dispute rating 10 4.9 
Feared retaliation from administration or appraiser for disputing rating 33 16.2 
Request to appeal rating was ignored 21 10.3 
Not familiar with appeals process 11 5.4 
Wanted to dispute rating, but chose not to 24 11.8 
Experience with TADS: 
Appraisals are subjective: administration or appraiser 17 8.3 
Appraisals are subjective: student population 3 1.5 
Appraiser did not adhere to timeline 1 0.5 
The quality of TADS is highly dependent on appraiser 8 3.9 
The reason for the rating received was not transparent or unclear 7 3.4 
TADS is not designed to measure effectiveness of ancillary or non-core subject teachers 5 2.5 
Unable to evaluate response 31 15.2 


Source: TADS Evaluation Survey, 2016-2017 

Notes: This question was presented to teachers that responded “Yes” to the question, “At any point in time, have you wanted to dispute 
your ratings and/or Student Performance measures?” (n=735). “Not applicable” response variations were removed from the 
total teacher responses for this item. (n=21). This question was presented to teachers only. 


HISD Research and Accountability 55 


TADS SURVEY ANALYSIS, 2016-2017 


Table F-22. Sample Teacher Responses to the Open-Ended Item, “Please use the space below to add any additional information about 


VolULa=y.dol-Val-later-MUU UM aleM-1e) el-t-liom ol cele-tt Mm tare] MMOLUm alia @uolel(om ol: Mal-1|oyielmcoM intl ocon diate Mm Uil-M eo) cele(-\-t- rads 


Sample Suggestions to Improve the Appeals Process 


“Going through an appeals process is an anxious period of time. Everyone going through this should have a mentor or advocate to help them 
through it.” 

“‘[The] appeals process made my ratings go lower. | was said to have been combative and resistant every time | wanted to dispute my ratings or 
get a different appraiser for inaccurate scores and/or ratings. The appraisers need supervisors other than the principal to assure they are rating 
teachers accurately and fairly.” 

“The fact that there's only one appraiser seems pretty bogus. Also, teachers should be able to rate their appraiser at the end of the year the 
same way they rate us. Why should | be given a score by someone who was too busy to be in my classroom or even provide help/feedback? 
The people on the bottom should always be able to appraise their leaders. Not being able to do this creates a culture of absolute power where 
teachers feel they can't defend themselves or dispute anything.” 

“[I] am not familiar with the process. | feel like we should've been told our admin how to do this during TADS training and | know we were not. | 
feel like we should also get an appraisal tool to rate our appraisers/administrators. We are evaluated to make sure we are doing our job right but 
| have never been asked about how | feel about how my admin is doing.” 


Sample Experiences with the Appeals Process or TADS 


“| had a productive discussion with my appraiser, and we amicably agreed on the topic of concern. There was no need to appeal.” 

“When | was called in to sign my final observation/end of the year paperwork | was not asked to review any of the ratings. | was to simply accept 
them. The more | thought about it, the more courage | got to ask for a review and his answer was ‘that time has come and gone!” 

“TADS is a biased system that is subjective and unfair. It is all based on one person's opinion and not a fair accurate assessment. If a principal 
likes you, you get a good rating and if they do not like you, your scores drop. The rubric basically makes it impossible to receive a 4, which is 
frustrating and discouraging. Why put forth the effort if you know you won't succeed?” 

“| responded to administrator's observations, but did not receive a response back from the administrator. | have not been thoroughly trained in 
the use of the TADS accountability system. | will spend time this summer reviewing all components and meet with my new administrator to 
determine his/her vision for my classroom performance and then document what | perceive is expected from me.” 

“In the 2014-2015 school year, | experienced administrative bullying. | was too afraid of retaliation by my appraiser to indicate any dissatisfaction 
with the process, and therefore | did not request a second appraiser or appeal. How can teachers feel safe to make realistic requests to protect 
their livelihoods?” 

“| did not appeal my evaluation because | was afraid of my appraiser taking it personally and making things difficult for me in the future, | feel 
more confident now and will not let that happen again” 

“| decided that it was not worth the effort. My overall scores were great, but there were a few categories in which | felt | deserved a higher score. 
My appraiser indicated that appraisers are not allowed to give too many fours.” 


Total Teacher Respondents 204 


Source: TADS Evaluation Survey, 2016-2017 
Note: This question was presented to teachers that responded “Yes” to the question, “At any point in time, have you wanted to dispute 
your ratings and/or Student Performance measures?” (n=735). 
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Table F-23. Experience with TADS Tools, 2016-2017 


Teacher Appraiser 
Total number of survey responses 3,538 435 
SP Tool FD Tool SP Tool FD Tool | 
Number of users who experienced technical difficulties 636 18% 331 9% 211 49% 156 36% 
Category of difficulty 

Ilost access to the SP online tool. 194 31% 94 28% 103 49% 74 47% 

The system timed out. 282 44% 132 40% 141 67% 117 75% 

There is missing data (e.g., forms, ratings). 281 44% 114 34% 129 61% 63 40% 

|was unable to locate data that | had previously saved. 289 45% 133 40% 127 60% 77 | 49% 

Other (please specify) 140 22% 65 20% 35 17% 18 12% 

The most recent technical difficulty | had was serious 121 19% 47 14% 62 29% 44 28% 
Location and internet connection when experiencing a technical difficulty 

lwas at my campus or another HISD location. 534 84% 272 82% 187 89% 135 87% 

Iwas off-campus (e.g., at home). 103 16% 61 18% 62 29% 58 37% 

lwas connected to an Ethernet cord. 217 34% 114 34% 87 41% 62 40% 

Iwas connected to wifi. 416 65% 208 63% 152 72% 117 75% 

Iwas using Google Chrome. 413 65% 197 60% 137 65% 99 63% 

Iwas using Internet Explorer. 313 | 49% | 167 50% 137 65% 103 | 66% 

Number of users who submitted a ServiceDesk ticket 89 14% 43 138% 68 32% 47 30% 

The particular technical difficulty was never resolved 16 18% 5 12% 8 12% 8 17% 

The issue was resolved in a timely manner 50 56% 26 60% 33 49% 20 43% 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
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Did you receive any formal support through online resources (i.e., houstonisdpsd.org) or 
online training (i.e. the HUB) during the 2016-2017 school year?/Did you recommend or 
require that a teacher you appraise receive formal suport through online resources (i-e., 
houstonisdpsd.org) or online training (i.e. the HUB) during the 2016-2017 school year? 


General instructional needs (e.g., assessment, planning and pacing, differentiation). 
Subject or content specific instructional needs (e.g., teaching fractions ina math class) 
Classroom management needs (i.e., organizing a classroom to maximize learning). 


Teacher Respondents Only 


Based on your experience with online resources or online trainings this year, how much of 
this type of support do you HISD should provide? 


laccessed this type of support one time only. 


Yes 


Teachers Appraisers 


1,819 75.8 203 69.8 


Teachers Appraisers 


82.6 


1,411 181 


145 


90.5 


1,242 73.4 


72.9 


1,091 64.6 157 77.7 


None of this type of support for me. 


Less of this type of support for me. 


The same amount of support for me. 


More of this type of support for teachers 
like me. 


Strongly Disagree Disagree 


Le) 


Teachers 


582 24.2 


Teachers 


188 10.6 
923 52.2 
595 33.7 


Neutral 
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Appraisers 


88 30.2 | 2,401 100.0 291 100.0 


Appraisers 


Agree 


Teachers 


Teachers 


1,709 
1,692 
1,689 


Total 
Appraisers 


Total 


Appraisers 


100.0 


100.0 


100.0 


Strongly Agree 


This type of support was a good use of my time. 79 45 144 8.1 366 20.6 981 55.3 203 11.5 | 1,773 | 100.0 
limplemented something from this type of support over the long term. 65 3.7 123 7.0 366 20.8 998 56.6 212 12.0 | 1,764 | 100.1 
This type of support directly helps me to meet the academic needs of my students. 15) 42 122 6.9 392 22.1 983 (s}3)i3) i}9)8) 112 legal 99.9 
This type of support is tied to my needs as identified in my performance appraisal. 100 5.7 193 10.9 482 27.3 837 474 153 8.7 1,765 | 100.0 
Source: TADS Evaluation Survey, 2016-2017 

Note: Percentages may not equal 100 due to rounding. 
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Table F-25. TADS Evaluation Survey Reponse on Formal Supports: Professional Learning Communities (PCLs), 2016-2017 


Yes No Total 
Teachers Appraisers Teachers Appraisers Teachers Appraisers 


Yes 


Le} Total 
Teachers Appraisers Teachers Appraisers Teachers Appraisers 


Did you receive any formal support through Professional Learning Communities (PLCs) 
during the 2016-2017 school year?/Did you recommend or require that a teacher you 
appraise receive formal suport through Professional Learning Communities (PLCs) during 
the 2016-2017 school year? 


General instructional needs (e.g., assessment, planning and pacing, differentiation). 1,450 91.5 200 


181 


96.6 


Subject or content specific instructional needs (e.g., teaching fractions in a math class) 1,324 83.8 88.3 


Classroom management needs (i.e., organizing a classroom to maximize learning). 1,239 78.8 175 84.5 


Teacher Respondents Only 


None of this type of support for me. 


Less of this type of support for me. 171 10.6 
Based on your experience with Professional Learning Communities (PLCs) this year, how 


much of this type of support do you HISD should provide? 
The same amount of support for me. 931 57.5 


More of this type of support for teachers 


fike ine: 459 28.4 
Strongly Disagree Disagree Neutral Agree Strongly Agree 
laccessed this type of support one time only. 330 20.5 582 
This type of support was a good use of my time. ay/ 3.5 113 7.0 277 17.2 936 58.1 228 14.2 1,611 | 100.0 
limplemented something from this type of support over the long term. 35 2.2 74 46 245 15.2 1,001 62.3 253 15.7 | 1,608 | 100.0 
This type of support directly helps me to meet the academic needs of my students. 41 25) 82 oul 274 16.9 961 59.4 259 16.0 1,617 99.9 
This type of support is tied to my needs as identified in my performance appraisal. 64 4.0 120 7.5 354 22.0 846 52.6 225 14.0 | 1,609 | 100.1 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
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Table F-26. TADS Evaluation Survey Reponse on Formal Supports: Teacher Development Specialist (TDS), 2016-2017 


Yes No Total 
Teachers Appraisers Teachers Appraisers Teachers Appraisers 


Did you receive any formal support through working with a Teacher Development Specialist 
(TDS) during the 2016-2017 school year?/Did you recommend or require that a teacher you 879 463 119 64.0 1900 100.0 186 100.0 
appraise receive formal suport through working with a Teacher Development Specialist , : : 
(TDS) during the 2016-2017 school year?* 


Total 


Teachers Appraisers Teachers Appraisers Teachers Appraisers 


General instructional needs (e.g., assessment, planning and pacing, differentiation). 
Subject or content specific instructional needs (e.g., teaching fractions in a math class) 722 84.8 
Classroom management needs (i.e., organizing a classroom to maximize learning). 


Teacher Respondents Only 


None of this type of support for me. 


Less of this type of support for me. 70 8.3 
Based on your experience with this year with a Teacher Development Specialist (TDS), how 


much of this type of support do you HISD should provide? 
The same amount of support for me. 432 51.4 


More of this type of support for teachers 


like me. we Gay 


Strongly Disagree Disagree Neutral Agree Strongly Agree Total 


laccessed this type of support one time only. 217 25.5 


This type of support was a good use of my time. 22 2.6 39 46 125 14.7 471 Doo) 191 22.5 848 2.) 
limplemented something from this type of support over the long term. 15 1.8 29 3.4 120 14.2 484 57.1 200 23.6 848 100.1 
This type of support directly helps me to meet the academic needs of my students. 20 24 26 3.1 124 146 479 56.3 202 23.7 851 100.1 
This type of support is tied to my needs as identified in my performance appraisal. 26 3.1 38 45 163 19.2 447 52.7 175 20.6 849 100.1 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
*A total of 465 teachers and 104 appraisers reported that a TDS was not available at their campus. 
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Table F-27. TADS Evaluation Survey Reponse on Formal Supports: Campus-based Mentor or Campus-based Teacher Leader, 2016-2017 


Yes LX Te) Total 
Teachers Appraisers Teachers Appraisers Teachers Appraisers 
Did you receive formal support through working with a campus-based mentor or campus-based 
Teacher Leader (e.g., a Career Pathways participant) during the 2016-2017 school year?/Did you 
recommend or require that a teacher you appraise receive formal suport through working with a 584 28.9 180 71.4 2,024 100.0 
campus-based mentor or campus-based Teacher Leader (e.g., a Career Pathways participant) during 
the 2016-2017 school year? 
Total 
Teachers Appraisers Teachers Appraisers Teachers Appraisers 


166 
153 


93.8 


General instructional needs (e.g., assessment, planning and pacing, differentiation). 


Subject or content specific instructional needs (e.g., teaching fractions in a math class) 486 85.4 86.0 


Classroom management needs (i.e., organizing a classroom to maximize learning). 499 88.6 167 92.8 


Teacher Respondents Only 


None of this type of support for me. 


Less of this type of support for me. 29 Sa 
Based on your experience working with a campus-based mentor or campus-based Teacher Leader 


this year, how much of this type of support do you think HISD should provide? 
The same amount of support for me. 308 54.3 


More of this type of support for teachers 


like me. aie Sel 


Strongly Disagree Disagree Neutral Strongly Agree Total 


laccessed this type of support one time only. 


This type of support was a good use of my time. 14 2.5 15 2.6 He 12.7 330 58.1 137 24.1 568 100.0 
limplemented something from this type of support over the long term. 12 2A 8 1.4 65 11.5 340 60.0 142 25.0 567 100.0 
This type of support directly helps me to meet the academic needs of my students. 16 2.8 9 1.6 67 11.8 333 58.5 144 25.3 569 100.0 
This type of support is tied to my needs as identified in my performance appraisal. 16 2.8 15 2.7 79 13.9 324 57.0 134 23.6 568 100.0 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
*A total of 328 teachers and 37 appraisers reported that a Campus-based Mentor or Campus-based Teacher Leader was not available at their campus. 
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Table F-28. TADS Evaluation Survey Reponse on Formal Supports: School Administrator, 2016-2017 


Yes No Total 
Teachers Appraisers Teachers Appraisers Teachers Appraisers 
Did you receive any formal support through working with an administrator at the school (including 
yourself) on instructional practice during the 2016-2017 school year?/Did you recommend or require 1214 51.7 234 81.0 1134 483 2348 100.0 289 100.0 
that a teacher you appraise receive formal suport through working with an administrator at the school , ; , 
(including yourself) on instructional practice during the 2016-2017 school year? 
Total 
Teachers Appraisers Teachers Appraisers Teachers Appraisers 


91.3 227 


209 


97.8 100.0 


100.0 


General instructional needs (e.g., assessment, planning and pacing, differentiation). 1,086 1,190 


Subject or content specific instructional needs (e.g., teaching fractions in a math class) 947 80.0 90:5 1,184 


Classroom management needs (i.e., organizing a classroom to maximize learning). 1,009 85.2 217 93.5 1,185 100.1 


Teacher Respondents Only 


None of this type of support for me. 


Less of this type of support for me. 85 V2 
Based on your experience working with an administrator at your school on your instructional practice 


this year, how much of this type of support do you think HISD should provide? 
The same amount of support for me. 727 61.6 


More of this type of support for teachers 


eqn. 332 28.1 


Strongly Disagree Disagree Neutral Agree Strongly Agree 


laccessed this type of support one time only. 


This type of support was a good use of my time. 26 2.2 43 3.6 155 13.1 738 62.4 221 18.7 | 1,183 | 100.0 
limplemented something from this type of support over the long term. 16 1.4 29 25 156 13.2 767 64.8 215 18.2 | 1,183 | 100.0 
This type of support directly helps me to meet the academic needs of my students. 24 2.0 31 2.6 163 13.7 744 62.6 226 19.0 | 1,188 | 100.0 
This type of support is tied to my needs as identified in my performance appraisal. 25 2.1 37 3.1 181 15.3 726 61.4 214 18.1 1,183 | 100.0 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
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Table F-29. TADS Evaluation Survey Reponse on the Quality of Formal Supports Offered Through TADS, 2016-2017 


The formal supports that | need to improve my instructional practice are available to me./The 
types of formal supports provided to me have helped me to better support teachers in their 
instructional practice this year.* 


The types of formal supports provided to me have helped me to improve my instructional 
practice this year./The formal supports that | need to improve my support of teachers’ 
instructional practice are available to me during the school year.* 


I spend an appropriate amount of time on professional development outside the duty 


day.(both) 


spend an appropriate amount of time on professional development inside the duty day.(both) 


Appraiser Respondents Only 


As anappraiser, what supports did you receive this year? 


Strongly Disagree 


Teachers 

135 5.8 7 
135 | 5.8 §) 
115 5.0 74 
144 62 12 


My SSO 
My PCIM 


My principal mentor 
Other leaders at my 
school 
Other campus-based 
leaders not at my 
Instructional Rounds 


Face-to-face district- 
wide training 
A manager support 
group 


Online Training 


Appraisers 


24 


3.1 


24 


4.2 


106 
105 


Disagree 


Teachers 


218 «9.4 
170 7.4 
229, 9.9 
247 = 10.7 


36.8 
36.5 
53.5 
28.5 
30.6 
45.1 

1.0 
26.4 


Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
*Indicates a proxy question used for a domain. 
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Appraisers 


29 


43 


57 


52 


10.1 


15.0 


19.8 


18.1 


Neutral 


Teachers 


436 


514 


425 


444 


18.9 


22.3 


18.4 


19.3 


Appraisers 


56 19.5 
65) | 226 
51 17.7 
49 17.0 
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Agree 


Teachers 


1,227 53.1 


1,208 52.3 


1,166 50.5 


1,194 51.8 


63 


Appraisers 


165 


139 


143 


154 


57.5 


48.4 


49.7 


53.5 


Strongly Agree 


Teachers 


294 12.7 
Be Be 
372 16.1 
277 | 12.0) 


Appraisers 


30 10.5 
31 10.8 
30 10.4 
21 7.3 


Teachers 


2,310 


100.0 


2,309 100.0 


2,307 100.0 


2,306 100.0 


Total 


Appraisers 


287 


287 


288 


288 


100.0 


100.0 


100.0 


100.0 


Table F-30. TADS Evaluation Survey Reponse on Retention, 2016-2017 


What is your best estimate for how long you plan to remain as a teacher in 
HISD? 


TADS SURVEY ANALYSIS, 2016-2017 


Five years or more after 2016-2017 


Between three and five years after 
2016-2017 


Fewer than three years after 2016-2017 


lam leaving HISD before the 2017-2018 
school year 
Strongly 


. Neutral 
Disagree 


Disagree 


1,324 57.6 


451 19.6 
372 «16.2 
151 66 


Strongly 
Agree 


Students’ progress and achievements make teachers feel valued. 200 8.7 | 230 10.0 | 340 14.7 |1,054 45.7 | 485 21 | 2,309 100.0 
Teachers are supported and fairly recognized by their school leaders. 354 153) 367 159] 4389 19.0] 860 37.3] 288 12 | 2,308 100.0 
Teachers have opportunities to progress professionally within the district. 217 94 | 286 124) 497 216 | 998 43.3 | 306 13 | 2,304 100.0 
HISD's appraisal system accurately documents teachers’ strengths and 391 17.0] 408 177 | 542 2351780 3391] 181 8 2302 100.0 
strategies to develop professionally. 
Teachers and teacher teams support each other in their work. 142 6.2 | 159 69 338 14.7 | 1,168 50.6 | 500 22 | 2,307 100.0 
Source: TADS Evaluation Survey, 2016-2017 
Note: Percentages may not equal 100 due to rounding. 
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